Metrics
The SDK Metrics enables clients to collect useful runtime usage metrics and unknown domains to share with Olfeo to improve the service provided.
As telemetry is collected on each Categorizer instance, it is agregated and stored in your Redis database, and then sent to Olfeo through the update service.
Runtime usage
The SDK's runtime usage data is automatically collected and reported when using the database updater. This data solely includes the number of categorization requests made and whether they succeeded.
Note
We do not collect which queries were performed.
Unknown domains
The SDK indentifies and collects domains that were queried but not found in the database. This allows seamless and automatic reporting to Olfeo's calssification team for continuous improvement.
Collection of data customization
You can customize or completely disable data collection by implementing the following interface: When the following interface is implemented : gitlab.olfeo.tech/data tools/nexus/sdk/metrics.Recorder
type Recorder interface {
// AddCategorizeCount records the result of a categorization process
AddCategorizeCount(ctx context.Context, resultStatus ResultStatus, queryType QueryType, outputType OutputType)
// AddCategoryCount records the category returned, if any, from categorization
AddCategoryCount(ctx context.Context, categoryId uint32)
// AddApplicationsCount records the number of application ids returned from a categorization
AddApplicationsCount(ctx context.Context, applications int)
// AddUnknownDomain record the unknown domain
AddUnknownDomain(ctx context.Context, domain string)
}it to the Categorizer at creation time, like this:
import "gitlab.olfeo.tech/data-tools/nexus/sdk/categorize"
type NullRecorder struct {}
func (n NullRecorder) AddCategorizeCount(_ context.Context, __ metrics.ResultStatus, ___ metrics.QueryType, ____ metrics.OutputType) {}
// Implement other methods of metrics.Recorder ...
func createCategorizer() {
...
categorizer, err := categorize.NewCategorizer(db, categorize.WithRecorder(NullRecorder{})
if err != nil {
...
}
...
}