Showcase the potential instrumentation workflows and solicit feedback.
What is Next
We will be focused on completing centralization of repos and handling of bespoke datasets.
Next steps/followup items:
Setting up time with Product Analytics to start walking through this, giving them access to test it and generate feedback. This is not a "hard commit" but we do need something we can poke holes in, versus talking about intangibles.
Talking about sanitization by fields with data engineering & adding to our backlog.
Drafting documentation & communications.
Looking at potential other events we would want to add as samples. We have a short list that Mikhail has helped inform, and will also touch base with Maya on this.
With bespoke data, is it possible to add new content to JSON blobs or bespoke dimensions map without changing the schema?
Yes - that's the goal. Must have column, must have type, type can't change, but we want the values to change & make it structured. So in the string case, it's really a piece of structured data that's been serialized into a string. To work on it as a piece of structured data, you would un-serialize the string.
In cases with dimensions and measures, where you've chosen a type in advance, there's flexibility with adding new keys and changing properties. There's an essential amount of freedom to vary the content. With bespoke data we would have standard fields, and a "flexibility area" which allows us to do things engineering-wise that we can't do today. All of Metrics Platform's events, even though they're structured differently and we have different quality controls/processes in place, we fundamentally have the same shape/same schema. That allows us to do nice things with data integration, and allows for a sandbox for "non-standardized" data.
In other words: changes to what bespoke data that instruments collect will not require event schema changes.