Skip to main content
Version: 1.3.0

Rest Emitter

The Rest emitter can be used to push metadata to DataHub.

​

class datahub.emitter.rest_emitter.DataHubRestEmitter(gms_server, token = None, timeout_sec = None, connect_timeout_sec = None, read_timeout_sec = None, retry_status_codes = None, retry_methods = None, retry_max_times = None, extra_headers = None, ca_certificate_path = None, client_certificate_path = None, disable_ssl_verification = False, openapi_ingestion = None, client_mode = None, datahub_component = None, server_config_refresh_interval = None)

Bases: Closeable, Emitter

  • Parameters:
    • gms_server (str)
    • token (Optional[str])
    • timeout_sec (Optional[float])
    • connect_timeout_sec (Optional[float])
    • read_timeout_sec (Optional[float])
    • retry_status_codes (Optional[List[int]])
    • retry_methods (Optional[List[str]])
    • retry_max_times (Optional[int])
    • extra_headers (Optional[Dict[str, str]])
    • ca_certificate_path (Optional[str])
    • client_certificate_path (Optional[str])
    • disable_ssl_verification (bool)
    • openapi_ingestion (Optional[bool])
    • client_mode (Optional[ClientMode])
    • datahub_component (Optional[str])
    • server_config_refresh_interval (Optional[int])

close()​

  • Return type:None

emit(item, callback=None, emit_mode=EmitMode.SYNC_PRIMARY)​

emit_mce(mce)​

emit_mcp(mcp, async_flag=None, emit_mode=EmitMode.SYNC_PRIMARY, wait_timeout=datetime.timedelta(seconds=3600))​

emit_mcps(mcps, emit_mode=EmitMode.SYNC_PRIMARY, wait_timeout=datetime.timedelta(seconds=3600))​

emit_usage(usageStats)​

fetch_server_config()​

Fetch configuration from the server if not already loaded.

  • Return type:RestServiceConfig
  • Returns: The configuration dictionary
  • Raises:ConfigurationError – If there’s an error fetching or validating the configuration

flush()​

  • Return type:None

get_server_config()​

  • Return type:dict

invalidate_config_cache()​

Manually invalidate the configuration cache.

  • Return type:None

property server_config : RestServiceConfig​

test_connection()​

  • Return type:None

to_graph()​

​

datahub.emitter.rest_emitter.DatahubRestEmitter()

alias of DataHubRestEmitter

​

class datahub.emitter.rest_emitter.EmitMode(value, names = None, *, module = None, qualname = None, type = None, start = 1, boundary = None)

Bases: ConfigEnum

ASYNC = 'ASYNC'​

ASYNC_WAIT = 'ASYNC_WAIT'​

SYNC_PRIMARY = 'SYNC_PRIMARY'​

SYNC_WAIT = 'SYNC_WAIT'​

​

class datahub.emitter.rest_emitter.RequestsSessionConfig(**data)

Bases: ConfigModel

  • Parameters:
    • data (Any)
    • timeout (float | Tuple [float,float] | None)
    • retry_status_codes (List [int])
    • retry_methods (List [str])
    • retry_max_times (int)
    • extra_headers (Dict [str,str])
    • ca_certificate_path (str | None)
    • client_certificate_path (str | None)
    • disable_ssl_verification (bool)
    • client_mode (ClientMode | None)
    • datahub_component (str | None)

build_session()​

  • Return type:Session

ca_certificate_path : Optional[str]​

client_certificate_path : Optional[str]​

client_mode : Optional[ClientMode]​

datahub_component : Optional[str]​

disable_ssl_verification : bool​

extra_headers : Dict[str, str]​

classmethod get_client_mode_from_session(session)​

Extract the ClientMode enum from a requests Session by checking the headers.

  • Parameters:session (Session) – The requests.Session object to check
  • Return type:Optional[ClientMode]
  • Returns: The corresponding ClientMode enum value if found, None otherwise

model_config : ClassVar[ConfigDict] = {'_schema_extra': <function ConfigModel.Config._schema_extra>, 'extra': 'forbid', 'ignored_types': (<class 'cached_property.cached_property'>,), 'json_schema_extra': <function ConfigModel.Config._schema_extra>}​

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

retry_max_times : int​

retry_methods : List[str]​

retry_status_codes : List[int]​

timeout : Union[float, Tuple[float, float], None]​

​

class datahub.emitter.rest_emitter.RestSinkEndpoint(value, names = None, *, module = None, qualname = None, type = None, start = 1, boundary = None)

Bases: ConfigEnum

OPENAPI = 'OPENAPI'​

RESTLI = 'RESTLI'​

​

datahub.emitter.rest_emitter.preserve_unicode_escapes(obj)

Recursively convert unicode characters back to escape sequences

  • Parameters:obj (Any)
  • Return type:Any