Configuration

Add appropriate configuration to the app.

AWS_REGION

AWS region

required; string; AWS only

AWS_ACCOUNT_ID

AWS account id

required; string; AWS only

AWS_ACCESS_KEY

AWS access key

required; string; AWS only

AWS_CONNECT_TIMEOUT_S

AWS connection timeout

optional; int; default: 2; AWS only

AWS_ENDPOINT_SNS

AWS endpoint for SNS. This may be used to customized AWS endpoints to assist with testing, for example, using localstack.

optional; string; AWS only

AWS_ENDPOINT_SQS

AWS endpoint for SQS. This may be used to customized AWS endpoints to assist with testing, for example, using localstack.

optional; string; AWS only

AWS_READ_TIMEOUT_S

AWS read timeout

optional; int; default: 2; AWS only

AWS_SECRET_KEY

AWS secret key

required; string; AWS only

AWS_SESSION_TOKEN

AWS session token that represents temporary credentials (for example, for Lambda apps)

optional; string; AWS only

GOOGLE_APPLICATION_CREDENTIALS

Path to the Google application credentials json file. If running in Google Cloud, these is automatically managed by Google. If passed in explicitly, this is respected, and takes precedence. See Google Cloud Auth docs for further details.

optional; string; Google only

GOOGLE_CLOUD_PROJECT

The Google Project ID that contains Pub/Sub resources. This is automatically interpreted based on the credentials, however it may be overridden if Pub/Sub resources live in a different project.

optional; string; Google only

GOOGLE_PUBSUB_READ_TIMEOUT_S

Read from PubSub subscription timeout in seconds

optional: int; default: 5; Google only

HEDWIG_CALLBACKS

A dict of Hedwig callbacks, with values as callables or fully-qualified function names. The key is a tuple of message type and major version pattern of the schema.

required for consumers; dict[tuple[string, string], string]

HEDWIG_CONSUMER_BACKEND

Hedwig consumer backend class

required; string

HEDWIG_DATA_VALIDATOR_CLASS

The validator class to use for schema validation. This class must be a sub-class of hedwig.validators.HedwigBaseValidator, and may add additional validation logic. This class is also responsible for serialization / deserialization of the payload on the wire.

Validators provided by the library: jsonschema, protobuf, protobuf-json.

To customize jsonschema validator, for example, to add a new format called vin, use this validator:

class CustomValidator(hedwig.validators.JSONSchemaValidator):
    # simplistic check: 17 alphanumeric characters except i, o, q
    _vin_re = re.compile("^[a-hj-npr-z0-9]{17}$")

    @staticmethod
    @hedwig.validators.JSONSchemaValidator.checker.checks('vin')
    def check_vin(instance) -> bool:
        if not isinstance(instance, str):
            return True
        return bool(CustomValidator._vin_re.match(instance))

optional; fully-qualified class name; defaults to “hedwig.validators.jsonschema.JSONSchemaValidator”

HEDWIG_DEFAULT_HEADERS

A function that may be used to inject custom headers into every message, for example, request id. This hook is called right before dispatch, and any headers that are explicitly specified when dispatching may override these headers.

If specified, it’s called with the following arguments:

default_headers(message=message)

where message is the outgoing Message object, and its expected to return a dict of strings.

It’s recommended that this function be declared with **kwargs so it doesn’t break on new versions of the library.

optional; fully-qualified function name

HEDWIG_MESSAGE_ROUTING

A dict of Hedwig message types, with values as topic names. The key is a tuple of message type and major version pattern of the schema. An entry is required for every message type that the app wants to publish. For publishing cross-project topic messages, instead of topic name, use: - AWS - a tuple of topic name and AWS account id (must exist in the same region) - Google - a tuple of topic name and GCP project id

It’s recommended that major versions of a message be published on separate topics.

required; dict[tuple[string, string], Union[string, Tuple[string, string]]]

HEDWIG_PRE_PROCESS_HOOK

A function which can used to plug into the message processing pipeline before any processing happens. This hook may be used to perform initializations such as set up a global request id based on message headers. If specified, this will be called with the following arguments for AWS SQS apps:

pre_process_hook(sqs_queue_message=sqs_queue_message)

where sqs_queue_message is of type boto3.sqs.Message.

For Lambda apps as so:

pre_process_hook(sns_record=record)

where sns_record is a dict of a single record with format as described in lambda sns format.

For Google apps as so:

pre_process_hook(google_pubsub_message=google_pubsub_message)

where google_pubsub_message is of type google.cloud.pubsub_v1.subscriber.message.Message.

It’s recommended that this function be declared with **kwargs so it doesn’t break on new versions of the library.

optional; fully-qualified function name

HEDWIG_PROTOBUF_SCHEMA_MODULE

The name of the module representing the Hedwig protobuf schema. This module must be pre-compiled and must contain all messages. Each message type’s schema must include all valid major versions.

required if using protobuf; string; fully-qualified module path

HEDWIG_POST_PROCESS_HOOK

Same as HEDWIG_PRE_PROCESS_HOOK but executed after message processing.

HEDWIG_PUBLISHER

Name of the publisher

required for publishers; string

HEDWIG_PUBLISHER_BACKEND

Hedwig publisher backend class

required; string

HEDWIG_PUBLISHER_GCP_BATCH_SETTINGS

Batching configuration for the GooglePubSubAsyncPublisherBackend publisher.

See Google PubSub Docs for more information.

optional; google.cloud.pubsub_v1.BatchSettings; Google only

HEDWIG_QUEUE

The name of the hedwig queue (exclude the HEDWIG- prefix).

required; string

HEDWIG_JSONSCHEMA_FILE

The filepath to a JSON-Schema file representing the Hedwig schema. This json-schema must contain all messages under a top-level key schemas. Each message’s schema must include all valid versions for that message.

required if using json schema; string; filepath

HEDWIG_SUBSCRIPTIONS

List of all the Hedwig topics that the app is subscribed to (exclude the hedwig- prefix). For subscribing to cross-project topic messages, instead of topic name, use a tuple of topic name and GCP project id.

required; List(Union(string, Tuple[string, string])); Google only

HEDWIG_SYNC

Flag indicating if Hedwig should work synchronously. If set to True a published message will be dispatched immediately using HEDWIG_CALLBACKS without calling any SQS APIs. This is similar to Celery’s Eager mode and is helpful for integration testing. It’s assumed that your service handles the message you’re dispatching in sync mode.

optional; bool; default False

HEDWIG_USE_TRANSPORT_MESSAGE_ATTRIBUTES

Flag indicating if meta attributes should be sent as transport message attributes. If set to False, meta attributes are sent as part of the payload - this is the legacy method for publishing metadata and newer apps should not change this value.

optional; bool; default True.