# Leveraging Common Schema ### About this export | Field | Value | | --- | --- | | **content_type** | lesson | | **platform** | contentstack-academy | | **source_url** | https://www.contentstack.com/academy/courses/data-insights-data-ingestion-profile-construction/data-insights-course-3--leveraging-common-schema | | **course_slug** | data-insights-data-ingestion-profile-construction | | **lesson_slug** | data-insights-course-3--leveraging-common-schema | | **markdown_file_url** | /academy/md/courses/data-insights-data-ingestion-profile-construction/data-insights-course-3--leveraging-common-schema.md | | **generated_at** | 2026-04-28T06:55:44.146Z | > Part of **[Data Ingestion & Profile Construction](https://www.contentstack.com/academy/courses/data-insights-data-ingestion-profile-construction)** on Contentstack Academy. **Academy MD v3** — structured for retrieval; no quiz or assessment keys. #### Video details #### At a glance - **Title:** 11-data-insights-leveraging-common-schema - **Duration:** 6m 46s - **Media link:** https://cdn.jwplayer.com/previews/IpTB9DvQ - **Publish date (unix):** 1752871658 #### Streaming renditions - application/vnd.apple.mpegurl - audio/mp4 · AAC Audio · 113582 kbps - video/mp4 · 180p · 180p · 148044 kbps - video/mp4 · 270p · 270p · 171185 kbps - video/mp4 · 360p · 360p · 186101 kbps - video/mp4 · 406p · 406p · 201597 kbps - video/mp4 · 540p · 540p · 249425 kbps - video/mp4 · 720p · 720p · 324626 kbps - video/mp4 · 1080p · 1080p · 551509 kbps #### Timed text tracks (delivery) - **thumbnails:** `https://cdn.jwplayer.com/strips/IpTB9DvQ-120.vtt` #### Transcript All right, so let's pull up one thing that I like to show folks is just some sort of as you start to play with the system, Lytx comes with a set of what we call the common schema. It's a set of attributes that you can use out of the box that don't require any mappings, any maintenance, any sort of configuration. Definitely always recommend that folks start with those and just sending data into the fields that exist. So I like to call it a couple of things that are really easy to use as you start to play with it. And then once you kind of get the hang of stuff, you can go look at your scheme and see what all is available. So if I go back over here. Let's pull, make sure we're on the right profile. Okay, so this is our profile on the left hand side. We'll make this a little bit bigger because it is probably teeny tiny. So we went through sort of just like first name, last name, email. As I'm playing with accounts, two really, really useful fields that are kind of special in our predefined schema or common schema is one, this attribute. So you can send, and we should do this within reason. You shouldn't send like 10,000 different attributes to be very clear. But as you're starting to play with it, you're just like, I want to send data into the system so that I can build an audience and see how it works. There's this sort of special attribute field where if you prefix a value with HTR underscore and then send a string, it'll actually add it to a map on the profile. So for instance, if I go in here and say, I'm going to just send the attribute underscore example as testing, I'm going to copy and paste that, send it as that user. And then over here, if I refresh and go to my profiles. So you'll see this field called custom user attributes. And in this case, we pass example as value testing, you can then actually build segments and do some sort of playing with that particular piece of data with no mapping, no configuration, nothing necessary. It works very similarly to some of our event and actions. So for instance, if I pass event open, it's going to map that to an email open and so on and so forth. But as an example, if we then go into the mapping for this attribute, just to see how it's defined, how we're able to essentially be smart enough to just pull if it matches that prefix, it gets mapped, we'll go into our schema, we'll go into fields, and we'll search for attribute. So if I pull up this field, which is not an identity field, so it's not going to merge profile data together, it's just going to surface information that you can segment, ultimately leverage it, one is a map string string, so it's going to take it and create a map of the information that you pass. And then we talked about merge operators, in this case, it's doing the merge operator of merge, which just means it's going to continue to push that object together, it's not going to fully overwrite it. So if I push a new key of like, example two, it's just going to add it to that object and combine the two together. So it's not going to overwrite it. If you open this up, there's a number of merge operators. So like I was mentioning, in the case of say, like, I want to know the first item that a user ever purchased, maybe you want to keep the oldest value of that. If I want to know the latest product that they purchased, you can keep the latest. Same thing with latest map and oldest map. So the different field types have different merge operators. I don't know if it's worth walking through, like all of the different options, they're all in our docs. But I would say as those things come up, and you'll see it kind of come up in some of the examples that we'll build, the merge operator is really important, and that it tells the system how to sort of combine that data together. Merge, merge is the one that I always like to touch on, because it's probably poorly named in the long run. But it's what sort of like keeps the old data and merges it with the new data that comes in so that you're not wholesale overwriting the object. So you've been doing or we've been using the JavaScript tag to collect data, which just uses our collection API. But there are a number of different ways that you can ultimately pull data into the system. So one of which is just our API. So for instance, I have just a simple, very simple JSON file over here, that's just an array of two objects. If I wanted to just push that information to a particular stream, and again, I'll share this document out so that you have some of the API calls, but I can just essentially fire a POST request to our collection API. The last parameter is the stream that it's going to pass to. So it's important to make sure that whatever stream you're passing data to has the mappings in place necessary to ultimately surface that on the profile. One major gotcha that a lot of folks run into is if you collect the data before it's mapped, it's not going to retroactively surface that data. We do have the concept of a replay or a rebuild. It's sort of I always like to describe it as like the last, the last ditch effort and that like if a customer were to go and goof up their data or mess up their mappings and need to essentially replay all of their events, we have the ability to do that from scratch to fix some of the issues that they run into. But it's a thing that one, can be very expensive, two, we don't like to do it very often. So it's important to think about before you actually collect data, where are you sending it and making sure that the mappings are in place to translate those raw events to the fields. Otherwise, it'll just sit in your stream and never get used. Could you just walk through the stream functionality stream once actually? And like streams just a tag that gets tagged onto the event that comes in. It's commonly just used to give you an idea of what the source it's from. So like if the data is coming from like Salesforce, the data from Salesforce will be in the Salesforce stream. If it's coming from the web, it goes to the default stream, which is typically web data. So it's really just a way to say, oh, this data from this source, I want to treat with these mappings, which are how the, it's kind of like the ETL process of how we convert that stream into fields. Yep. #### Subtitles (WebVTT) ```webvtt WEBVTT 1 00:00:00.000 --> 00:00:21.620 All right, so let's pull up one thing that I like to show folks is just some sort of 2 00:00:21.620 --> 00:00:26.680 as you start to play with the system, Lytx comes with a set of what we call the common 3 00:00:26.680 --> 00:00:27.680 schema. 4 00:00:27.680 --> 00:00:31.440 It's a set of attributes that you can use out of the box that don't require any mappings, 5 00:00:31.440 --> 00:00:34.360 any maintenance, any sort of configuration. 6 00:00:34.360 --> 00:00:37.680 Definitely always recommend that folks start with those and just sending data into the 7 00:00:37.680 --> 00:00:38.680 fields that exist. 8 00:00:38.680 --> 00:00:42.840 So I like to call it a couple of things that are really easy to use as you start to play 9 00:00:42.840 --> 00:00:43.840 with it. 10 00:00:43.840 --> 00:00:45.820 And then once you kind of get the hang of stuff, you can go look at your scheme and 11 00:00:45.820 --> 00:00:48.120 see what all is available. 12 00:00:48.120 --> 00:00:54.640 So if I go back over here. 13 00:00:54.640 --> 00:01:03.280 Let's pull, make sure we're on the right profile. 14 00:01:03.280 --> 00:01:09.040 Okay, so this is our profile on the left hand side. 15 00:01:09.040 --> 00:01:13.400 We'll make this a little bit bigger because it is probably teeny tiny. 16 00:01:13.400 --> 00:01:16.080 So we went through sort of just like first name, last name, email. 17 00:01:16.080 --> 00:01:20.920 As I'm playing with accounts, two really, really useful fields that are kind of special 18 00:01:20.920 --> 00:01:25.400 in our predefined schema or common schema is one, this attribute. 19 00:01:25.400 --> 00:01:28.720 So you can send, and we should do this within reason. 20 00:01:28.720 --> 00:01:32.300 You shouldn't send like 10,000 different attributes to be very clear. 21 00:01:32.300 --> 00:01:34.840 But as you're starting to play with it, you're just like, I want to send data into the system 22 00:01:34.840 --> 00:01:37.720 so that I can build an audience and see how it works. 23 00:01:37.720 --> 00:01:43.560 There's this sort of special attribute field where if you prefix a value with HTR underscore 24 00:01:43.560 --> 00:01:47.760 and then send a string, it'll actually add it to a map on the profile. 25 00:01:47.760 --> 00:01:52.880 So for instance, if I go in here and say, I'm going to just send the attribute underscore 26 00:01:52.880 --> 00:01:59.240 example as testing, I'm going to copy and paste that, send it as that user. 27 00:01:59.240 --> 00:02:05.040 And then over here, if I refresh and go to my profiles. 28 00:02:05.040 --> 00:02:07.760 So you'll see this field called custom user attributes. 29 00:02:07.760 --> 00:02:11.920 And in this case, we pass example as value testing, you can then actually build segments 30 00:02:11.920 --> 00:02:17.640 and do some sort of playing with that particular piece of data with no mapping, no configuration, 31 00:02:17.640 --> 00:02:19.400 nothing necessary. 32 00:02:19.400 --> 00:02:24.520 It works very similarly to some of our event and actions. 33 00:02:24.520 --> 00:02:29.340 So for instance, if I pass event open, it's going to map that to an email open and so 34 00:02:29.340 --> 00:02:30.520 on and so forth. 35 00:02:30.520 --> 00:02:36.360 But as an example, if we then go into the mapping for this attribute, just to see how 36 00:02:36.360 --> 00:02:40.960 it's defined, how we're able to essentially be smart enough to just pull if it matches 37 00:02:40.960 --> 00:02:47.080 that prefix, it gets mapped, we'll go into our schema, we'll go into fields, and we'll 38 00:02:47.080 --> 00:02:51.120 search for attribute. 39 00:02:51.120 --> 00:02:55.440 So if I pull up this field, which is not an identity field, so it's not going to merge 40 00:02:55.440 --> 00:02:59.320 profile data together, it's just going to surface information that you can segment, 41 00:02:59.320 --> 00:03:04.980 ultimately leverage it, one is a map string string, so it's going to take it and create 42 00:03:04.980 --> 00:03:08.160 a map of the information that you pass. 43 00:03:08.160 --> 00:03:12.720 And then we talked about merge operators, in this case, it's doing the merge operator 44 00:03:12.720 --> 00:03:16.680 of merge, which just means it's going to continue to push that object together, it's not going 45 00:03:16.680 --> 00:03:17.680 to fully overwrite it. 46 00:03:17.680 --> 00:03:24.080 So if I push a new key of like, example two, it's just going to add it to that object and 47 00:03:24.080 --> 00:03:25.080 combine the two together. 48 00:03:25.080 --> 00:03:26.720 So it's not going to overwrite it. 49 00:03:26.720 --> 00:03:30.240 If you open this up, there's a number of merge operators. 50 00:03:30.240 --> 00:03:35.240 So like I was mentioning, in the case of say, like, I want to know the first item that a 51 00:03:35.240 --> 00:03:39.400 user ever purchased, maybe you want to keep the oldest value of that. 52 00:03:39.400 --> 00:03:44.240 If I want to know the latest product that they purchased, you can keep the latest. 53 00:03:44.240 --> 00:03:45.920 Same thing with latest map and oldest map. 54 00:03:45.920 --> 00:03:50.560 So the different field types have different merge operators. 55 00:03:50.560 --> 00:03:54.520 I don't know if it's worth walking through, like all of the different options, they're 56 00:03:54.520 --> 00:03:56.000 all in our docs. 57 00:03:56.000 --> 00:03:58.840 But I would say as those things come up, and you'll see it kind of come up in some of the 58 00:03:58.840 --> 00:04:03.320 examples that we'll build, the merge operator is really important, and that it tells the 59 00:04:03.320 --> 00:04:05.720 system how to sort of combine that data together. 60 00:04:05.720 --> 00:04:09.760 Merge, merge is the one that I always like to touch on, because it's probably poorly 61 00:04:09.760 --> 00:04:12.680 named in the long run. 62 00:04:12.680 --> 00:04:16.280 But it's what sort of like keeps the old data and merges it with the new data that comes 63 00:04:16.280 --> 00:04:22.080 in so that you're not wholesale overwriting the object. 64 00:04:22.080 --> 00:04:27.240 So you've been doing or we've been using the JavaScript tag to collect data, which just 65 00:04:27.240 --> 00:04:30.240 uses our collection API. 66 00:04:30.240 --> 00:04:35.520 But there are a number of different ways that you can ultimately pull data into the system. 67 00:04:35.520 --> 00:04:39.040 So one of which is just our API. 68 00:04:39.040 --> 00:04:43.800 So for instance, I have just a simple, very simple JSON file over here, that's just an 69 00:04:43.800 --> 00:04:45.560 array of two objects. 70 00:04:45.560 --> 00:04:50.840 If I wanted to just push that information to a particular stream, and again, I'll share 71 00:04:50.840 --> 00:04:54.840 this document out so that you have some of the API calls, but I can just essentially 72 00:04:54.840 --> 00:04:57.920 fire a POST request to our collection API. 73 00:04:57.920 --> 00:05:01.440 The last parameter is the stream that it's going to pass to. 74 00:05:01.440 --> 00:05:06.560 So it's important to make sure that whatever stream you're passing data to has the mappings 75 00:05:06.560 --> 00:05:11.440 in place necessary to ultimately surface that on the profile. 76 00:05:11.440 --> 00:05:16.680 One major gotcha that a lot of folks run into is if you collect the data before it's mapped, 77 00:05:16.680 --> 00:05:20.600 it's not going to retroactively surface that data. 78 00:05:20.600 --> 00:05:24.520 We do have the concept of a replay or a rebuild. 79 00:05:24.520 --> 00:05:29.640 It's sort of I always like to describe it as like the last, the last ditch effort and 80 00:05:29.640 --> 00:05:33.320 that like if a customer were to go and goof up their data or mess up their mappings and 81 00:05:33.320 --> 00:05:37.680 need to essentially replay all of their events, we have the ability to do that from scratch 82 00:05:37.680 --> 00:05:40.120 to fix some of the issues that they run into. 83 00:05:40.120 --> 00:05:43.960 But it's a thing that one, can be very expensive, two, we don't like to do it very often. 84 00:05:43.960 --> 00:05:48.640 So it's important to think about before you actually collect data, where are you sending 85 00:05:48.640 --> 00:05:52.640 it and making sure that the mappings are in place to translate those raw events to the 86 00:05:52.640 --> 00:05:53.640 fields. 87 00:05:53.640 --> 00:05:56.240 Otherwise, it'll just sit in your stream and never get used. 88 00:05:57.120 --> 00:06:03.280 Could you just walk through the stream functionality stream once actually? 89 00:06:03.280 --> 00:06:08.320 And like streams just a tag that gets tagged onto the event that comes in. 90 00:06:08.320 --> 00:06:14.040 It's commonly just used to give you an idea of what the source it's from. 91 00:06:14.040 --> 00:06:19.080 So like if the data is coming from like Salesforce, the data from Salesforce will be in the Salesforce 92 00:06:19.080 --> 00:06:20.080 stream. 93 00:06:20.080 --> 00:06:26.160 If it's coming from the web, it goes to the default stream, which is typically web data. 94 00:06:27.080 --> 00:06:30.000 So it's really just a way to say, oh, this data from this source, I want to treat with 95 00:06:30.000 --> 00:06:34.520 these mappings, which are how the, it's kind of like the ETL process of how we convert 96 00:06:34.520 --> 00:06:36.520 that stream into fields. 97 00:06:36.520 --> 00:06:37.520 Yep. ``` ```transcript [00:00] All right, so let's pull up one thing that I like to show folks is just some sort of [00:21] as you start to play with the system, Lytx comes with a set of what we call the common [00:26] schema. [00:27] It's a set of attributes that you can use out of the box that don't require any mappings, [00:31] any maintenance, any sort of configuration. [00:34] Definitely always recommend that folks start with those and just sending data into the [00:37] fields that exist. [00:38] So I like to call it a couple of things that are really easy to use as you start to play [00:42] with it. [00:43] And then once you kind of get the hang of stuff, you can go look at your scheme and [00:45] see what all is available. [00:48] So if I go back over here. [00:54] Let's pull, make sure we're on the right profile. [01:03] Okay, so this is our profile on the left hand side. [01:09] We'll make this a little bit bigger because it is probably teeny tiny. [01:13] So we went through sort of just like first name, last name, email. [01:16] As I'm playing with accounts, two really, really useful fields that are kind of special [01:20] in our predefined schema or common schema is one, this attribute. [01:25] So you can send, and we should do this within reason. [01:28] You shouldn't send like 10,000 different attributes to be very clear. [01:32] But as you're starting to play with it, you're just like, I want to send data into the system [01:34] so that I can build an audience and see how it works. [01:37] There's this sort of special attribute field where if you prefix a value with HTR underscore [01:43] and then send a string, it'll actually add it to a map on the profile. [01:47] So for instance, if I go in here and say, I'm going to just send the attribute underscore [01:52] example as testing, I'm going to copy and paste that, send it as that user. [01:59] And then over here, if I refresh and go to my profiles. [02:05] So you'll see this field called custom user attributes. [02:07] And in this case, we pass example as value testing, you can then actually build segments [02:11] and do some sort of playing with that particular piece of data with no mapping, no configuration, [02:17] nothing necessary. [02:19] It works very similarly to some of our event and actions. [02:24] So for instance, if I pass event open, it's going to map that to an email open and so [02:29] on and so forth. [02:30] But as an example, if we then go into the mapping for this attribute, just to see how [02:36] it's defined, how we're able to essentially be smart enough to just pull if it matches [02:40] that prefix, it gets mapped, we'll go into our schema, we'll go into fields, and we'll [02:47] search for attribute. [02:51] So if I pull up this field, which is not an identity field, so it's not going to merge [02:55] profile data together, it's just going to surface information that you can segment, [02:59] ultimately leverage it, one is a map string string, so it's going to take it and create [03:04] a map of the information that you pass. [03:08] And then we talked about merge operators, in this case, it's doing the merge operator [03:12] of merge, which just means it's going to continue to push that object together, it's not going [03:16] to fully overwrite it. [03:17] So if I push a new key of like, example two, it's just going to add it to that object and [03:24] combine the two together. [03:25] So it's not going to overwrite it. [03:26] If you open this up, there's a number of merge operators. [03:30] So like I was mentioning, in the case of say, like, I want to know the first item that a [03:35] user ever purchased, maybe you want to keep the oldest value of that. [03:39] If I want to know the latest product that they purchased, you can keep the latest. [03:44] Same thing with latest map and oldest map. [03:45] So the different field types have different merge operators. [03:50] I don't know if it's worth walking through, like all of the different options, they're [03:54] all in our docs. [03:56] But I would say as those things come up, and you'll see it kind of come up in some of the [03:58] examples that we'll build, the merge operator is really important, and that it tells the [04:03] system how to sort of combine that data together. [04:05] Merge, merge is the one that I always like to touch on, because it's probably poorly ``` #### Key takeaways - Connect **Leveraging Common Schema** back to your stack configuration before moving to the next module. - Capture one concrete artifact (screenshot, Postman call, or code snippet) that proves the step works in your environment. - Re-read the delivery versus management boundary for anything you changed in the entry model. ## Supplement for indexing ### Content summary Leveraging Common Schema. Leveraging Common Schema in Data Ingestion & Profile Construction (data-insights-data-ingestion-profile-construction). ### Retrieval tags - Leveraging - Common - Schema - data-insights-data-ingestion-profile-construction - lesson 03 - Leveraging Common Schema - data-insights-data-ingestion-profile-construction lesson ### Indexing notes Index this lesson as a primary chunk tagged with lesson_id "03" and topics: [Leveraging, Common, Schema]. Parent course slug: data-insights-data-ingestion-profile-construction. Use asset_references URLs as thumbnail hints in search results when present. Never surface LMS quiz content or assessment answers from this file. ### Asset references | Label | URL | | --- | --- | | Video thumbnail: Leveraging Common Schema | `https://cdn.jwplayer.com/v2/media/IpTB9DvQ/poster.jpg?width=720` | ### External links | Label | URL | | --- | --- | | Contentstack Academy home | `https://www.contentstack.com/academy/` | | Training instance setup | `https://www.contentstack.com/academy/training-instance` | | Academy playground (GitHub) | `https://github.com/contentstack/contentstack-academy-playground` | | Contentstack documentation | `https://www.contentstack.com/docs/` |