The modern data experience

We’ve spilled a lot of ink on the modern data stack. We talk about its architecture, which tools fit where, and how to organize our teams to support it. As the data community, it’s easy to be excited about this impressive new technology.

But for most people, the modern data stack isn’t a collection of architectural diagrams; it’s an experience. It’s rushing to answer your CEO’s urgent question before this afternoon’s exec meeting; it’s spending more time trying to verify that a dashboard is accurate than talking about what the dashboard actually says; it’s piecing together this month’s board deck from disconnected BI reports, SQL queries, and Excel files, a task that you swore you did last month.

We don’t have these issues because our technology is coming up short; we have these issues because we need to pair an experiential roadmap with our architectural one. In this talk, I’ll walk through examples of what that roadmap might look like—and propose how we can all move from building the modern data stack to creating the modern data experience.

Follow along in the slides here.

Browse this talk’s Slack archives #

The day-of-talk conversation is archived here in dbt Community Slack.

Not a member of the dbt Community yet? You can join here to view the Coalesce chat archives.

Full transcript #

[00:00:00] Anna Filippova: All right. Who else is having the best day ever? I know I am my name’s Anna. I’m heading up community here at dbt Labs and I’m just blown away by the sessions today. I’m hosting this session and Mila Paige who’s our esteemed developer advocate is your chat champion today at #coalesce-mode. Of course, the title of the session is Metrics and the modern data experience.

And we’re joined today by the one and only Benn Stancil. You might recognize Benn from the classic Friday data fights on Twitter or references to Substack. Yes, the very same ones that are on the bingo board and his spare time of which he has a lot. He’s also the co-founder and chief analytics officer at Mode .

Things you need to know about Benn . He will fight you about coffee cake. He’s from one of the Carolinas. Can you guess which one? And he will always have a better metaphor. Than you that’s just facts. So today’s talk is part of a conversation that Benn and a lot of other folks in the community have been having for several months. Now you heard from Drew a little while ago [00:01:00] on metrics, actually a lot on metrics and 5,000 years of metrics.

And Jeremy, just before this session shared how all of this is going to work in one point now, and I’m very excited today to see Ben bringing all of this together. A gentle reminder that all chat conversations are happening in #coalesce-mode. The memes are already starting. Don’t miss out. It’s on the dbt community Slack.

We’re going to share relevant links in Slack as Benn is talking. So you won’t miss any of the references. And of course, slides are going to be available with an email that Lauren’s going to put together at the end of the day. Please do say hi, introduce yourself in the Slack. Do ask other attendees questions, make comments, react at any point in the channel and do use Slack threads to keep the conversation tidy.

After the session, Benn is going to hang around and answer some questions so throw them in the chat anytime, and we’ll make sure to come back to them. Oh, and one. It’s not Friday yet, but the best hot take during today’s session is going to unlock a very special hot take back [00:02:00] from Benn. Let’s get started over to you, Benn.

[00:02:05] Benn Stancil: All right. How do so I just saw, Drew’s talk for the first time, a couple hours ago. So aside from the screenshots that I took, that you made. Many similarities between this aside, including our apparent shared paralyzing fear of snakes are entirely coincidental. Just a disclaimer on that.

Okay. So a couple days ago at the start of this conference, Tristan posed a question along the lines of what are we going to be talking about today that we’re going to remember in a hundred years? And so like on December 8th, 2121 hard date to say what are we still gonna be talking about?

Like any of this stuff still. I don’t know if we’re going to remember today a hundred years from now. We’ve got a lot of infamous history to compete with on December 7th tonight. But I do think that today will matter. And I think specifically it will matter because of what drew just announced that, that announcement marks the end of an era and the beginning of another one.

So on this like decades long or millennia long journey the data journey that we’re on, I think today actually marks a rare discontinuity. So in other words basically I want to say that the metrics layers the future of analytics in [00:03:00] order to do that I’m going to follow a classic John Oliver narrative arc.

I want to talk about how we got here. I want to talk about what’s changed with what you announced. And I want to talk about where we going to go next. And so to do that I want to talk about initially to start with where it all be. Which is the ancient Egyptians. But I’ve done. I started by his sense of history is not nearly as good as Drew’s.

So my history actually starts in 1989. So 1999 micro strategy was founded and a year later in 1990 business objects was founded. And back before this was back before business objects was SAP business objects, it was acquired by SAP and back before micro strategy was a Bitcoin company.

These two companies were pioneering BI companies and they ushered in what I would call the age of big BI where there was big guy tools, these big monolithic BI platforms that really changed the way that we think about data and roughly here’s what they did. Companies would have data that was stored in a database.

They would store wall. It was messy represented by these messy purple tables here. But it wasn’t just a database. It was stored in a slow and expensive database. So people couldn’t run a lot of queries against it. There wasn’t that much you could do that was. [00:04:00] And BI tools came along and figure out a way to fix this.

So I did it in a couple of ways first because that data was messy. BI tools developed the way to model that mess. And these were the models that would define relationships between tables. This is what that actually looks like in microstrategy. And this step would turn messy data like Stripe logs or things like that into something like dimension purchases, which is a clean data set where you have one row per order, you have which item was bought, how much it cost was it bought on a gift card, these sorts of things.

And then you would have tools that would define how to actually calculate metrics on top of that clean data. So you could define say revenue from dimension purchases, where you would add up the cost of every item, maybe remove things like store credits, remove taxes, all that. And this is again how you would define metrics in MicroStrategy.

And then on top of that, people would build reports and companies could suddenly do a lot more with their data. And so more stylistically, this is what it looked like. BI tools would end up taking, having modeling raw data and putting it into a data model out would come in these nice clean tables metrics would then get computed on top of that.

So clean [00:05:00] tables like dimension purchases could be turned into revenue and then people would start to put. Charts and reports on top of that now, as Drew alluded to, there are obviously a lot of limitations with this. Everything had to be pre-computed. It all had to be loaded into this data model. The things you could see were pretty narrow and the data had to be pretty small.

You also couldn’t really do a whole lot with this data, the data in the warehouse, because the warehouse was expensive and slow. So there wasn’t actually much value in this, but the big thing that BI. Is that one huge benefit, which companies had a really consistent and reliable view of their businesses that anybody could look at reports and get the information they needed to make decisions using well-governed data and well-defined metrics.

And so this was a tremendously valuable thing. And to explain why I want to talk about cubes but not this cube, a Rubik’s cube. But not this Rubik’s cube or this Rubik’s cube actually. But this Rubik’s cube. So this is a seven dimensional Rubik’s cube. This is something that people actually try to solve shout out to my former coworker non-math and also data scientists for being the third person in the world to [00:06:00] solve this thing.

I only 11 people ever have. But when you’re trying to solve a Rubik’s cube like this, you can’t actually do it obviously with a physical toy. There’s no seven dimensional toy that you can play with to try to. The way you have to solve it as code. And this is the code that, that defines a actually four dimensional Rubik’s cube.

So you can imagine what a seven dimensional one might look like. And so solving this problem is plenty hard enough as it is. But if there is a bug in your code or one of the faces gets messed up, or your computer program that is modeling it crashes, you’ve got a real problem because you can’t go back on the physical cube to say, Hey, what does this thing actually look like?

And if that code breaks. Then the cube is broken that a seven dimensional Rubik’s cube only exists in as data and code. And if that code doesn’t work or it changes unexpectedly, you can’t actually solve this problem. And so companies are like these Rubik’s cubes, all decisions we want to make depend on the metrics that are defined in this code.

And so if we want to make decisions about revenue, then we have to have good definitions of what revenue actors. And so there’s a lot of decisions that are all based [00:07:00] on our understanding of revenue for making decisions about which product variants, to build, making decisions about how much to charge for that product, about what to do when an unexpected crisis hits or need to evaluate our ML models to making sure we’re not going to, we’re going to avoid a catastrophe.

We have to be able to understand what revenue is. And so that is really what BI solved. And so we solved this problem decades ago. I We all, at least now have a shared understanding of revenue or the other metrics that define our business. And then around 2012, things started to change. So one of the big reasons is Tristen has talked about in a prior blog post is that Redshift came along and what this did is it turned our slow and expensive databases into ones that were.

And that enabled us to do a lot more with this data. And so in addition, to be able to continue to do reporting and things like that, we could also do analytics. We could do operational, have built operational tools. We could apply AI to this data. And of course, dbt came along and pulled out a lot of this data modeling into its own layer.

And so dbt ended up cleaning up a lot of this data representation. [00:08:00] And so this really ushered in this kind of second age of data, which I would call the age of the modern data stack. And this has been a tremendously beneficial thing. It’s mostly been very good. We get all these very powerful things that we can do with data that we couldn’t do.

And Jason Ganz highlighted a lot of this yesterday. The impact we could have with data and a post analytics engineering world, which is the same as a modern data stack world is way higher than it was before we can wait, make way smarter decisions. We can do a lot better about how we think about marketing our products.

We can be smarter about operations, such as managing, using AI to manage fleets of airplanes. All of this stuff is great. Powerful. But on one of the more fundamental problems on the problem of reliably defining this Rubik’s cube and making sure all sharing the same reality when we’re trying to solve these problems, we’ve taken one step forward and three.

And the reason for that is because these metrics are now scattered all over the place to every tool in this stack has to represent how have to figure out how they actually want to represent that Rubik’s cube. There is no central definition for that Rubik’s cube and it has [00:09:00] to live in every individual tool.

And so you see this across different tools in the stack. So Looker has look ML in node. We do this through a thing called definitions where it’s a way to represent what this, these metrics look like. Observability tools like big. I have metrics. AI tools like CCU have ways to define metrics.

And of course there is an holy number of Excel spreadsheets and SQL queries that live out in the ether that define these metrics as well. And as a result, what happens is all these tools to borrow Drew’s language. We end up arguing over which latitude we should be using to measure things. And so the companies that are using them end up playing this endless game of whack-a-mole trying to get everything done.

And so this was something that was put really well byAli Ghodsi the CEO of Databricks. And Databricks is building like one of the most technologically advanced data tools out there. And at future data, he said that despite all that we’ve built, despite how far we have come in, all of this technology, most of their customers are running around trying to figure out why somebody’s numbers, don’t match somebody. Else’s.

All their customers spend time bickering about whose [00:10:00] numbers are. And so really, despite all of this, we’re really just reliving 1989. Until metrics layer. And so over the last year, we’ve seen a lot of activity in this space, which has obviously accelerated further in the last month lookers last month announced the opening up of a universal semantic layer.

And then obviously today dbt joined the party with Drew’s talk. And so to me, what this, what Drew’s talk and really dbt’s commitment to this represent is that we’re now solidifying a new architecture that looks like this. This is the architecture of the future where data models and metrics are centrally governed.

That dimension purchases is not just universally accessible, but the metrics that make the formula for revenue are also universally accessible too. And so these definitions can be accessible in other tools that can be accessible in these four here, but it could also be accessible and things like data discovery and catalog products that can be accessible and monitoring tools and all of that.

Now what this does is this makes our lives easier. And that’s really nice. It represents the sort of standards of the modern data stack, where data and [00:11:00] metrics are modular they’re version controlled. It also can tolerate governance into a clean layer where governance lives in the entire left-hand side of this diagram and consumption lives on this kind of wide array.

But what it really does, like the real impact of this is it means all of these tools on the right can stop worrying about data governance and metric consistency. It means these tools on the right can focus on building great last mile experiences for improving whatever they were built to improve. So it means that the right side of this diagram no longer has to worry about what the Rubik’s cube looks like and can instead just focus on solving it.

And so what that means is metrics are the beginning of a platform. They are at the beginning of an operating system for the day. And in the talk that Tristan, Andrew had Martine talked about how data is going to be a trillion dollar market. And there’s a question of how we get there, and this is how we get there.

This is how we make everybody a data buyer where metrics and the things underneath it, our platform and the apps on top of it are experiences. And so that’s why I think today marks the end of the age of the modern data stack [00:12:00] and the beginning of the age of the modern data experience, where we’re thinking about building these experiences on top of the stack that we now have.

Now that doesn’t mean we’re done a long way from it, because as great as this diagram is which I really like this diagram blueprints aren’t experiences that this map is not the territory. And there’s a lot of questions about what exactly the experience of using metrics should be. And so there’s a ton of different tools out there.

And I know nothing about most of these. I know nothing about monitoring and discovery and operations day I tools. I do know stuff about these. So I want to focus on these two of BI and analytics, which I think of as like general data consumption and exploration. These are the tools you use.

When you have a question you want to be able to answer it. You want to be able to explore data, those kinds of things. This represents only a part of what data is useful for. I recognize that, but it’s an important part. And so I want to talk about how we would experience using metrics in this world. And so within these two boxes, two big questions stand out to me.

[00:12:54] How do we interact with metrics? #

[00:12:54] Benn Stancil: First. How do we interact with metrics. That most implementations of metrics are code oriented. [00:13:00] The thing that Drew demoed was code a lot of the other tools that are building the sorts of things are primarily code oriented. This makes sense. That’s how you make it accessible to a bunch of different other tools, but in order to make it consumable for other people, it has to be translated into some interface that probably isnt code.

And today the way we have historically built that, and especially in self-serve tools or BI tools is around the concept of tables and columns that you see this on the left of Tableau. You see these dimensions and measures, which are mostly representing tables and columns that are in Tableau. You see the same kind of paradigm and Looker.

You see the same paradigm in mode as well, that most of this is constructed around the notion of tables and columns. And similarly tables and columns are obviously very important in SQL where that is the primary nouns that, that. Metrics don’t really fit into this paradigm. Now, if you want to look at a metric, you may not care or even know what the underlying table is.

When Drew was demoing what he was demoing, the idea of looking up something like revenue. We don’t know what table that’s underneath that’s not the point. The point is the metric, the nail, and there is a metric, not a table. And so outside of [00:14:00] interfaces, within some metrics, there are tools. This is transform, which is also building metrics layer.

We don’t really have a model for understanding how do we interact with data? There’s building on metrics rather than tables. So we need to figure. The second thing we have to be able to do is how do we actually use metrics in different tools? Because while there are only two boxes here, there’s actually a lot of different ways that we consume data within even just these two boxes.

For instance, we’ve all got dashboards. We have data exploration tools. We’ve obviously got SQL tools. We’ve got notebooks, we’ve got data apps, all of these things. If you put them all together are different experiences for consumers. And so there’s zoomed in view of those two boxes. We have to be able to figure out how to metrics actually solve problems across all of these experiences, which raises the third question, which is not only how do we solve problems across these different tools, but how do we, or how do we solve problems and these tools, how to do it across the tools?

How do we do it, where you’re actually bouncing between different types of ways of interacting with data? Because really what happens if you’re using all these things is you’re not just using these things in isolation. You’re often finding yourself, hopping between them, [00:15:00] between going from dashboards to notebooks, going from SQL based analysis to a data app, to a visual analysis, all those sorts of things.

And when that happens, we often find yourself falling into the gaps in between. And so an example that probably lots of people are familiar with is you’ve got a dashboard in your BI tool. Something in that dashboard is down, say, signups here is not looking so good. So I’m angry, exec comes along and says, Hey, what’s going on?

You’ve got to figure that out. And it’s actually pretty hard to extend that work straight from the BI tool and the Looker case, for instance, Looker will give you the SQL that generated that dashboard. But it’s pretty hard to use. It’s this is no knock on Looker. It’s it’s a machine generated code.

It’s just not really designed for people. And and it’s difficult to be able to take this work and extend it easily into trying to figure out what happened with science. And what often happens when we need to do that is we ended up starting over opening up a blank sequel editor and saying, okay, let’s dive in into this.

And so to, follow on to what Erica Louie was talking about at the very first talk of this conference, this doesn’t really mean we’re building more knowledge. We’re really just repeating it over. [00:16:00] And so the question is, can the metrics layer turn this disjointed experience with these gaps between ways of working into something that’s more connected and is something where these things are actually more, more seamless.

And I think it’s worth trying to do this because if it does, there’s a really great consumption experience to be built on top of this. And so to me, what that experience could look like is people should be able to start from a library of metrics, choosing the ones they want to look at. Be able to choose the metric, choose a variety of details.

Say things like time dimensions, grouping filters, all the things that drew just showed for how to present that data, create data on top of it. So get results back. In some cases, those tables could be pretty wide. You want to be able to add multiple dimensions, add multiple metrics, derive metrics. Then you want to be able to visualize that result.

And this is more than simple charts. You want to be able to do it in pivot tables and rich visualizations and great ways that both understand that data and to present it. And then he would have to load those metrics into visualizations, into different outputs. It may be creating dashboards off of those metrics.

It may be creating narrative reports. It may be creating data apps[00:17:00] but you want to be able to do all sorts of. And this also shouldn’t just be for business users or people who aren’t wanting to write code that analysts should be able to follow the same workflow as well, either starting from a dashboard and then starting from scratch as well for the questions they have, they should be able to use metrics as a shortcut to get the sort of first level of answers they need to get.

And then when they get to a point where metrics no longer answer that question, they should be able to open up more complex analysis by easily extending the SQL and the queries that are generated by the metric. So they should be able to wrap those queries with their own SQL being able to enrich the metric, bringing a new model tables, bringing a new metrics, or even incorporating vault data.

If that’s what the question calls for. And then if analysts wanna be able to go further, they should go to open all of this up into notebooks for other rich environments, for being able to explore that data, using the type of deep statistical analysis that the analyser analysts. And finally Facebook would also create the same sorts of things.

Analysts aren’t just building one-off reports. They’re also building dashboards and things like that for sharing their workflows should facilitate that as well. [00:18:00] And if we’re able to build that this experience actually goes one step further, which is rather than just connecting these different ways of working.

We can blend them into one seamless experience because none of these modes are perfect. Like dashboards aren’t perfect visual analysis isnt perfect SQL analysis notebook. None of these are the perfect mode for working on any one problem. Sometimes we need code. Sometimes we need simple UIs. Sometimes we need rich visualization sometimes when you Python and R and we need to be able to move between these things depending on what problem we’re trying to solve or what step in solving that problem we’re actually in.

And so to me, this experience also merges BI and data. In a way where data science workflows can use the more powerful governance tools or attrition associated with BI and governance can be extended to answer questions that weren’t anticipated, where governance is typically more rigid. And so I think Martine also talked about the collision of these things.

This also facilitates that sort of collision, and also critically this isn’t persona based. So where people live in one tool and some other people live in another code, free [00:19:00] exploration workflow should be great shortcuts for analysts. As I said so long as they can be extended with the code when they need to be, and code rich analysis should be great for non analysts, as long as they can interact with the result as they would with a traditional BI tool.

So by having everybody be able to work together in this sort of way, we can turn an analytical experience into a melting pot and no place for the red people, for the blue people. And of course, for all of the purple people. And to me, that is the modern data experience. That is the next era of data consumption.

And so as the analyst who lives in these diagrams, this is the experience that I want to live in as the pointed here at exec who has to consume data and care about dashboards matching and things like that. And has a board report to, this is also the experience that I want to live here, but I am a third thing.

I’m also a founder of a data startup. And so I can also say this is experience. I want to be. And truthfully it’s the experience that I’ve wanted to build at mode for a long time. it, And it’s an experience that we have been building including for instance, we had a big launch of new visualizations just two days ago, if you haven’t seen it, check it out.

But we’ve wanted to build this and we haven’t been [00:20:00] able to, because in addition to building this outer ring, we’ve also had to focus a lot on what does the Rubik’s cube look like? We’ve had to define our own system of tons or of meters or of ways to measure things along with everybody else. And that is printed.

And so the metrics layer changes all this that we can now build on top of it and focus just on that experience of using metrics, as opposed to thinking about how do we govern them. And so over the last little while we’ve been talking with folks at dbt about their vision for the metrics layer and in those conversations, we’ve been thrilled to see how much it lines up with our vision, for where we want to go with things as well.

And so we believe with the platform that they’re creating with what. That there are things that we can do on top of that. So before I go I like drew have one more thing that I want to build a shape. I’m going to show you what it is that we’ve started to build. So within mode you’ll fin be able to create mode reports rather than just starting with a query.

Also starting with a metric, then people can choose to play from a list of defined metrics in that experience. They can choose the various properties later govern that [00:21:00] metric things like time dimensions to group by all those kinds of. They can choose the right filters to apply to that metric and then run it all without writing SQL, to be able to get these governed metrics that we’ve seen before, if they want to be able to visualize it, they can do that in those new visual Explorer, which like I said, was a big rewrite of our visualizations.

Check it out and create reports and dashboards on. And then if analysts want to follow the same flow they can, but if they want to extend it, they’ll also be able to open this up, see the query underneath it and see the rendered SQL that generates that metric. And if they want to use this as it is, they can, but also if they want to use these metric queries for building blocks with custom SQL wrapping that SQL directly around custom code, that’ll be possible as well.

And then this obviously all works and mode as it does today, where these queries can be extended with visualizations loaded into a notebook for further analysis and things like that. We’ll have a lot more to say about this in the coming months. In the meantime, if you’re interested in learning more, please reach out.

You can go to mode.com/coalesce. As soon as your email, I’ll also reach out over the dbt Slack. We’ll be happy to talk about it. As we progress, we’d love to get your [00:22:00] perspective on this your feedback on what you’d like to see and your take on whether or not the metrics are actually is the future of anyway.

So that I’ll stop there and hand it back over to Anna.

[00:22:12] Q&A #

[00:22:12] Anna Filippova: Wonderful. Thank you so much. Benn that was really awesome. And I’m very excited to see some questions rolling. I have one for you already, and that is, which is your favorite song. Can you guess, off of which elbow?

[00:22:29] Benn Stancil: There would be two to choose from. Good for you is the obvious answer from the Olivia and Rigo album on 1989.

I don’t know. I, I don’t know that I have, there’s a lot of good things there. I’m not actually as big of a fan of the Taylor’s versions of things. As perhaps I should be, or as my Spotify wrapped would tell me that I am so maybe I am, maybe the data is not line it doesn’t

[00:22:48] Anna Filippova: Yeah. The other question was do we all get free mode?

[00:22:51] Benn Stancil: If you want them to reach out, we can probably figure out a way to get some folks, some t-shirts.

[00:22:55] Anna Filippova: Yes. Folks are asking the important questions. I have a [00:23:00] question for you.

[00:23:01] Benn Stancil: Yes.

[00:23:04] Anna Filippova: What do you think we’re still missing?

[00:23:08] Benn Stancil: Like in the overall experience?

[00:23:11] Anna Filippova: Yeah. We talked a lot today. We’re all very excited, but obviously there’s more work left to do.

What’s the work that we have left to do.

[00:23:17] Benn Stancil: So I think that. This thing that I’ve just, there’s like a, I don’t know if you can hear this phone ringing in the background, I’ve noted this th that you don’t know what it is building these sorts of experience. I think there is like this kind of general consumption experience is important and it’s the way that people should be able to use data if they’re trying to answer questions.

So if you have a question you want to go answer that as Drew alluded to there is another way in which we consume data that. How do we consume it in the moment that we were actually trying to use it? What is the, what does it look like in Salesforce? What does it look like in a design tool?

What does it look like in a support tool? What does it look like in the embedded way? Not like an embedded dashboard, but embedded in the analogy I’ve used before is like. Where data’s embedded in our decision [00:24:00] about which restaurant we go to, how do we make a decision about, or how do we embed data in that way?

And I think there needs to be a platform for us to build that. On top of, to me, the metrics layer is the beginning of that. It is the beginning of using the data stack as a platform for building those sorts of apps. And those sorts of to me, that is what a data app actually is like Yelp as a data app, not a sort of drag and drop thing with.

But we’re getting close to that. And I think what y’all announced today is representing that, that sort of first step. But I do think there is stuff that’s missing to actually make a true platform to the point where we can all easily build those sorts of apps.

But it doesn’t, I think mean that they’re just BI or analytics or kind of the general consumption goes away. I think it just means that data gets consumed in a lot of different ways. And the kind of general consumption thing is always something that’ll be there.

[00:24:42] Anna Filippova: You mean BI is not dead long live BI.

[00:24:46] Benn Stancil: At this point, I don’t know if BI is dead or not. don’t even know what I think of. I don’t know what the last thing I said about it was I think it’s, I think it was good.

[00:24:53] Anna Filippova: I think it’s dead. We’ll see maybe Mila can pull up a link for us and throw it in the chat. Gregory has a question.

Gregory wants to know [00:25:00] if this is the first time that there are parameters for SQL table. Has this syntax been seen before?

[00:25:04] Benn Stancil: I’m sure that exists. I think that there are probably lots of people who have built internal versions of this with some sort of like ungodly mess of like stored procedures and things like that, where they’re probably stored procedures with functions and stuff in ‘em.

That’s not. I haven’t seen it in something that is so foundational. It’s often something, again, in the way, like metrics get defined in every tool. I’m sure there are lots of companies that define this kind of process and fairly narrow scopes. But I haven’t seen like the parameterized SQL query of sorts at the sort of scale that, that it potentially could be happening around this again, they’re there, I’m sure in all of these things, as I’m sure Drew was, he was going back and doing research for his talk. He found like tons of things that were like, oh my God, I can’t believe the world work that way.

If you were to go back and do research on the various like ins and outs of every BI tool and things like. There are so many different languages that people have built that probably solve all these problems. Like we are just repeating, solving those same problems over again. So I’m sure there is somewhere we can [00:26:00] find that has happened but I don’t think I’ve seen it in in a platform sort of way before.

[00:26:05] Anna Filippova: Yeah. Yeah. It’s like Microsoft got two touch screens in 2001 long before anyone needed one on a laptop. Here’s a hot take question for you. What is the slowest part of the data stack today?

[00:26:17] Benn Stancil: The slowest part. Like probably dbt. And it’s not so much like that like dbt’s compilation time. It’s slow. It’s the dbt does some of the hardest work. dbt takes a giant log of tables and says, Hey, transform this into something that is like well-governed and narrow. And that sort of thing. I don’t, that doesn’t mean it’s necessarily slow.

But there is a certain latency in that. There’s an, a latency in the ETL or the ELT step. Partly because you are bound by what the source can do. Like you can’t pull from Salesforce. I I think Fivetran is working on the sort of stuff, but you can’t pull from some places all the time because you have API limits and that sort of thing.

So overall it’s not that slow. The, honestly, probably the answer is the slowest part is like how quickly humans can react to it. And systems where it’s, [00:27:00] AI stuff and it needs to be, I’m gonna. Say something, I’m sure I’m going to start seeing ads for it in 15 seconds from now. But in other systems where you’re trying to make decisions like the humans building to make those willingness to make those decisions is often a lot slower than the data that gets loaded in the database.

[00:27:12] Anna Filippova: One last question, and then we’re going to wrap up. Blake wants to know is Mode is going to integrate with dbt metrics.

[00:27:17] Benn Stancil: We’ll have that conversation. Drew said to reach out we’ll reach out. Yeah.

Last modified on: