How to create a new data collection component
Component bootstrapping
Runedgee components new
to create a new data collection component and choose your preferred programming language.
This creates a local folder containing sample code and tools to help you get started quickly.
Component manifest file
Your component’s manifest file is namededgee-component.toml
. It contains all the information required to
build, test, and push your component via the Edgee CLI.
Components created via edgee components new
already include a default manifest you can easily customize.
In case you prefer managing the local folder and starting from a blank implementation, feel free to use
edgee components init
to initialize an empty manifest in the current folder.
A typical manifest file looks like this:
-
manifest-version
: it indicates the expected structure of your manifest file, the only available value for now is1
. -
[component]
: here you define what your component is all about, including its public name and latest versionname
: the name of this component, will be used for URLs and shown on public pages.slug
: the slug of this component, will be used for URLs and shown on public pages. Note: the slug is used as an identifier by the CLI (for a given organization). In other words, if you change a component’s slug, you will create a brand new component. A slug should change only when absolutely necessary.version
: the latest version of this component, will be used when pushing to update or create versions.language
: the programming language used this component, will be used to automate some language-specific build or compilation details. It’s automatically configured if you start with a starter template.category
: the only available value for data collection components isdata-collection
.subcategory
: the available values for now areanalytics
,warehouse
, andattribution
.description
: the description of this component, will be shown on public pages.documentation
: the documentation URL for this component (could also point to a public repository).repository
: the repository URL for this component (e.g. on GitHub).icon-path
: the image used on public pages to represent this component.wit-version
: the WIT interface implemented by this component. When a new WIT version is available, update this field to the latest version and the Edgee CLI will take care of updating all WIT dependencies for you.
-
[component.build]
: here you define how the component is built and where the resulting Wasm file is located.command
: the command to compile this component into Wasm, depends on your programming language of choice.output-path
: the local file path where your compiled Wasm binary is stored, will be used for local testing and pushing (make sure this is aligned with the build command defined above).
-
[component.settings.X]
: here you define as many settings as needed (see Reusability and settings below)title
: text that will be shown in the web console to identify this setting.type
: valid values arestring
,number
, orbool
.required
: (optional) a bool value to indicate whether this setting is required or not, false by default.description
: (optional) text that will be shown in the web console to describe this setting.options
: (optional) the allowed values for this setting.secret
: (optional) a bool value to indicate whether this setting value should be encrypted, false by default.
edgee components new
, most fields come with good defaults and you only need to customize
your component’s name, description, documentation URL, repository URL, and settings.
Implementing your component
A data collection component must implement the following WIT interface:page
, track
or user
event is sent for data collection
by the Edgee proxy. These methods receive the incoming event object and a set of settings.
They’re expected to return an edgee-request
object, which looks like this:
Note: Edgee components don’t perform HTTP calls directly, as they run in a sandboxed environment with no access
to network or file system. The intended purpose of a component is to instruct the proxy on which HTTP request to perform
to fullfil its data collection duties.For example, the Google Analytics component returns an
edgee-request
object that points to the official
https://www.google-analytics.com/g/collect
endpoint, with the correct querystring parameters.
Similarly, the Amazon S3 component points to https://{bucket}.s3.amazonaws.com
with the correct sigv4 authentication headers.Reusability and settings
To make your Edgee component reusable for other developers and organizations via the Edgee Component Registry, you can define a set of inputsettings
, containing all the API credentials or dynamic
information you need to craft the correct edgee-request
object. For example, the Google Analytics component only expects a
ga_measurement_id
setting, while the Segment component expects segment_project_id
and segment_write_key
.
You can configure the expected settings for your component by adding the following for each setting in your manifest file:
Note: only
title
and type
are mandatory to define a setting.
In this case, example-name
is the key your business logic will use to get its value from the settings
dict.Note: even if defined as
number
or bool
, all settings are treated as strings when your component receives them.
For now, you’ll need to parse them into the correct type. In the future, our WIT definitions will handle types automatically as well.Build and test locally
When the implementation is ready, compile your component into Wasm:edgee components new
the default build script
should be a great starting point. By default, the output of this command will be a new Wasm file in the current folder.
Note: the Edgee CLI is intended to simplify and make local development uniform across programming languages and component types.While some of the compiling details are still visible today because of long multi-step commands or additional support scripts,
our long-term goal is to hide most of that complexity behind
edgee components build
in the future.For example, you need to explicitly configure debug flags for your preferred language:- Rust components use
cargo build --release
by default, but you can enable debugging symbols by removing the--release
flag from the build command in your manifest file. (Please note that this will increase the size of your Wasm file considerably, up to 40x) - C components use
clang
to compile the Wasm file with the-Os
flag by default, but you can change this to-O0
in your manifest file. - Go components include debug symbols by default, but you can add the
-no-debug
flag to disable them and optimize the size and performance of your Wasm file.
--event-type
is provided, it will run the Wasm file with all event types (page
, track
, and user
).
This helps ensure your component behaves as expected from the proxy’s perspective, in addition to your unit tests.
Using test events
Test events represent the typical structure of an Edgee event. For example, the samplepage
event looks like the following:
Test events are static objects for now. In the near future, you’ll be able to provide a
custom JSON object that better suits your component’s needs.
Test HTTP requests
By default, the test command runs your local Wasm file and prints out the resultingedgee-request
object.
Once you’re happy with the implementation, you can bring your testing one step forward and use the CLI to run the
actual HTTP request for you:
Note: we recommend using test or staging environments whenever possible to avoid
logging test data into production systems or analytics APIs.
Push to the registry
When your component is ready for some action, it’s time to push it to the registry. Make sure you went through these steps so far:- Installed and configured the Edgee CLI
- Implemented the Edgee WIT interface
- Updated your manifest file with the correct build command and settings
- Validated the Wasm file with local tests
- Decided on the component’s visibility (public or private)
edgee components push ORG_ID
.
Congratulations! Your component is now available on the Edgee Component Registry at https://www.edgee.cloud/{organization}/{component}
.
In case you pushed a private component, its page is only visible to you.
You can also view and edit it at https://www.edgee.cloud/~/registry/{organization}
.
Publish/unpublish components
A public component is visible to anyone in the registry, while private components are only available to a specific organization. During the initial push, you decide between private or public. Every subsequent push command will keep the same visibility. If you need to publish a private component or unpublish a public component after the initial push, you’ll need to declare the visibility change explicitly:Note: unpublishing a public component is only possible as long as the component isn’t used by any project.
You can still push the same component as private for a specific organization. This might be useful for special
use cases where a component needs ad-hoc customizations.
Archive/delete components
After pushing, visithttps://www.edgee.cloud/~/registry/{organization}
and select your component to edit it.
Here you can archive or delete it with the corresponding buttons.
Please note that:
- Archived components become invisible on public pages, but can be unarchived. Existing projects using an archived component still work fine.
- Deleted components simply stop existing and this action is irreversible. If there is at least one Edgee project using a component, you cannot delete it.
Note: even though the web console allows you to edit most fields and creeate new versions, we recommend using the Edgee CLI
to push updates and new versions. This way, your manifest file remains the source of truth.
Use the component
Once your component is available via the Edgee Component Registry, you can add it to a project and it will start receiving events. First of all, you’ll need to create a project, configure it for your website, and then install the Edgee SDK. To simplify the setup, we recommend using the default Edgee-generated domain and keep routing settings as simple as possible. Visit the Components section of your project and click Add a component. Here you can filter by category, organization, and visibility. Make sure to select Private in the bottom-left if you pushed a private component. Select your component and configure it. A few things to keep in mind:- You can select a specific version of your component, by default the latest version is selected.
- Enable Auto update if you want to automatically use the latest version of your component for this project.
- If your component only support specific events, disable any unsupported event in the Event Activation section.