Pod Insights Practical AI Operationalizing ML/AI with MemSQL About this episode A lot of effort is put into the training of AI models, but, for those of us that actually want to run AI models in production, performance and scaling quickly become blockers. Nikita from MemSQL joins us to talk about how people are integrating ML/AI inference at scale into existing SQL-based workflows. He also touches on how model features and raw files can be managed and integrated with distributed databases.
Sponsors:
DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring:
Nikita Shamgunov – Website https://neon.tech/, X https://x.com/nikitabaseDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes:
MemSQL https://www.memsql.com/MemSQL’s ML/AI capabilities https://www.memsql.com/solutions/predictive-ml-ai/MemSQL’s recent AI/ML e-book https://www.memsql.com/resources/memsql-achieving-successful-operation-of-ml-and-aiContact tracing case study with MemSQL and True Digital https://www.memsql.com/blog/case-study-true-digital-group-helps-to-flatten-the-curve-with-memsql/Upcoming Events:
Register for upcoming webinars here https://practicalai.fm/webinars!
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at ChangeLaw because of Rollbar. Check them out at rollbar.com. And we're hosted on Linode cloud servers. Head to linode.com/changelog.
0:17 Speaker 00 This episode is brought to you by Digital Ocean. Digital Ocean's developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, TM accounts, worldwide availability with a 99.99 uptime SLA and 24/7 365 world-class support to back that up. Digital Ocean makes it easy to deploy, scale, store, secure, and monitor your cloud environments. Head to do.co/changelog to get started with a $100 credit. Again, do.co/changelog.
0:58 Speaker 01 Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.
1:24 Daniel Whitenack? Welcome to Practical AI. This is Daniel Whitenack. I'm a data scientist with S I L International. Normally I'd be joined by my co-host Chris Benson, who is a Principal AI Strategist at Lockheed Martin, but he's in the midst of some family health-related things. So he's taking the time that he needs, but we're definitely excited to chat about a really interesting topic today. Actually, in our Slack channel, I remember some conversation a couple of weeks ago where we were discussing the issue of, hey, I trained my model. It works great on my data. I evaluate and it all seems good. But then when I try to integrate this into code, the performance is actually really terrible.
2:08 Daniel Whitenack? And it's kind of a mismatch between production things. And I think that we're going to be able to get into some of those things today. Today we have as our guest Nikita Shamgunov, who's the CTO of MemSQL. Really excited to talk to you today. Nikita, welcome.
2:24 Speaker 03 Happy to be here. Like Daniel said, my name is Nikita. Actually, I'm co-CEO and founder of MemSQL. I don't mind the confusion. I started as the CTO and I took over as CEO in 2017.
2:37 Daniel Whitenack? Gotcha.
2:38 Speaker 03 And recently, about a year ago, brought a co-CEO Raj Verma with the thinking that we're going to take the company public.
2:46 Daniel Whitenack? Ah, gotcha. On that note, why don't you give a little bit of maybe the background first of yourself? And then we can get into maybe a little bit of the background of MemSQL. I think that would be great context.
2:58 Speaker 03 Definitely. So I've spent my career in data management and databases specifically. I came to the States, to the United States after finishing my grad school at St. Petersburg, in St. Petersburg, Russia, and joined the SQL Server Engine team. So went from kind of very research-oriented life and work to basically system engineering, right? When you build databases, it's actually a very different cadence versus using databases. When you use databases, you think about things like performance, you think about SQL as the API to the database, and then you think about reliability and uptime. And when you build a database, you think about quality.
3:41 Speaker 03 You think about the life of somebody who is using the database and how you make that lives easier. And obviously, you think about performance and scalability and how the database user or developer can achieve that performance and scalability.
3:56 Daniel Whitenack? It sounds like an interesting transition from the sort of academic world to the systems engineering world. Was it a hard shift for you or was that sort of focus on the user and reliability? Was that something that you were already kind of passionate about going into that work?
4:12 Speaker 03 I was very passionate about engineering in general. What I loved about building databases is that the product, the database engine, is like a computer science in a box. It has algorithms, it has data structures, it has system engineering, you interface with networking, I O, CPU caches, you need to be aware of the computer architecture in order to build, you know, world-class software. So that certainly resonated a lot. And that was the core premise of why I wanted to start working on, you know, world-class industry products. And then from there, the passion to the user came in and over time, just that curiosity about, you know, building new things and breaking ground and entrepreneurship
4:57 Speaker 03 came through the years while, you know, working at SQL Server. Mind you, during that time, Microsoft was going through a cloud transition. Everything that we're, you know, we're seeing today at scale, at that time, all of that stuff was being born, being conceived and major architectural choices were made. Some of them were right, some of them were not right. So that's my kind of a big company background. And then I switched and joined Facebook. And in fact, one of the premises of me joining Facebook was not to make a lot of money or because, you know, that was a pre-IPO, you know, 2010, right? Actually moved to the Silicon Valley and meet the kind of people I will later start a company with.
5:41 Speaker 03 And what happened is that as I walked into Facebook on day one, I met my future co-founder, Eric. And relatively shortly after, just within six to eight months, we started Meme SQL and I left Facebook.
5:56 Daniel Whitenack? Oh, that's a wild ride, I guess. Moving to a total new place, experiencing, you know, Facebook and that culture, especially at that sort of stage and then founding something. So maybe describe a bit how like that happened so fast. The idea from Meme SQL and the sort of motivation that this was something that was really needed. How did that occur?
6:16 Speaker 03 Yeah, so distributed systems at SQL Server, we always knew that was the future. Right? Especially as you go into the cloud transition. And the time, you know, you know, back in like 2008, 2010, Microsoft had a flagship product, SQL Server, which is a single node database system, very, very good, very, very powerful. Really proud I worked on this one. The main competitor, Oracle, had distributed systems at its disposal. Oracle Exadata and Oracle RAC. And the way the database market is structured is that the top tier workloads that have high performance requirements, high availability requirements, do require distributed systems and Microsoft didn't have that.
6:59 Speaker 03 And so that top of the market, Microsoft was losing to Oracle then. Actually, I think they mostly caught up right now. But then architecturally, single node databases are very hard to change and may turn them into a distributed system. That was some of the, you know, kind of moonshot projects that my co-founder and CTO Adam worked on at Microsoft. And that moonshot project didn't succeed. When I walked into Facebook, the need for distributed systems became apparent. Because every Facebook workload is that high level, high end workload. And sometimes it's from the reliability standpoint.
7:41 Speaker 03 Sometimes it's from the scale standpoint, most of the time the scale standpoint. Because, you know, back in 2010, I think Facebook was on the march to cross a billion active users. So that was on everyone's mind, that was everyone's goal. How can we cross a billion active users? And obviously, you know, history shows that Facebook had blown through that goal quite successfully.
8:02 Daniel Whitenack? Were they trying to architect something internally to deal with that? Or was it sort of an open problem when you were there?
8:10 Speaker 03 It's more than one system, right? For something that is so big, right? As Facebook, turns out that all the data workloads are split into categories. So some of them are data lakes and, you know, Hadoop basically got a lot of advancements at Facebook. Some of them are operational, you know, powering Facebook.com and multiple data management technologies are on the critical path between, you know, typing Facebook.com and actually seeing the news feed. There's a separate data management solution for messaging, separate news feed, and this goes on. And within that, also there's a whole bunch of kind of point solutions for various analytical workloads, right? One is for time series.
8:54 Speaker 03 In fact, there's a startup called Signal Effects, who took some of the ideas and then the folks from Facebook left, started that company that was recently acquired by Splunk.
9:03 Daniel Whitenack? I got you.
9:04 Speaker 03 And then there was a system called Scuba that gives you real time analytics and a lot of ideas there influence and sequel roadmap as well. So, long story short. Lots and lots of data management systems and data management workloads inside Facebook. But each and every one is a distributed system. And so that pervasiveness of distributed system, we're captivating. It like really validated the thinking that the future of database systems are is distributed. And that's how we started MemSQL.
9:35 Daniel Whitenack? Awesome.
9:36 Speaker 03 And we started this as a memory, hence the name.
9:38 Daniel Whitenack? Yeah.
9:39 Speaker 03 Now, in fact, the name is kind of limiting because MemSQL has evolved way past being in memory and memory only. It's the version one that was in memory and single node. But very quickly, we expanded to distributed system, built tiered architecture from memory to disk. And now we expanded it to S3 or, you know, other object stores.
10:01 Daniel Whitenack? Yeah, I'd be interested to hear a little bit about, I mean, you kind of gave a sense of the initial founding and some of the initial ideas. I'd be curious, as far as right now with MemSQL, could you just give a sort of high level view of the sorts of things that people are turning to MemSQL for? Kind of the consistent things that you see really people getting value out of? And then maybe some of the newer things that are enabling maybe new sorts of workloads that you didn't even anticipate in those early days?
10:29 Speaker 03 Definitely. So first of all, databases are very long game.
10:34 Daniel Whitenack? Yeah.
10:35 Speaker 03 And the most successful database products on the planets, which are Postgres, MySQL, SQL Server and Oracle, are all 30 plus years old.
10:44 Daniel Whitenack? Yeah.
10:44 Speaker 03 And we still use it today, which is basically if you turn into any other piece of technology, that's not the case.
10:51 Daniel Whitenack? Yeah.
10:51 Speaker 03 Right? Technology is very transient, right? We're building something and something new comes in and completely disrupts what's there before. But database seems to stay for a long time.
11:02 Daniel Whitenack? Yeah, I think in my experience from working at the different places, I've encountered Postgres a lot. I've encountered, you know, right now I'm working on a team that's using SQL Server for certain things. Of course, I've encountered certain things like, you know, Mongo or other databases that.
11:21 Daniel Whitenack? Or you know, like the NoSQL or those sorts of databases. But I think like you were talking about the user experience. It always seemed like to me the the natural user experience. And you gain a lot of power with that SQL interface to the database.
11:35 Speaker 03 The relational, yeah, for sure.
11:37 Daniel Whitenack? Yeah.
11:37 Speaker 03 And so, yeah, so the vision is a single pane of glass to all your data and all your workloads. But when you start peeling the layers and understanding what would it take to deliver on that vision? You start understanding how you scale storage, how you scale compute, and how you scale both storage and compute for your low latency operational workloads. Think about powering your apps and, you know, loading your webpage. And, you know, the results need to come back to you and ideally sub hundred milliseconds to running what is called big expensive analytical queries that scan large volumes of data to give you insights.
12:17 Speaker 03 Those insights could be reports. Those insights could be information, analytical information, which is also called decision support, right? You need to make a decision. So you need to know what works, what doesn't. You need to know, you know, how your sales are doing in this state versus the other state, this product versus that product. And so that is a continuous process of evaluating and looking at data and understanding, driving insights out of data. So the interesting piece about what I just described is that for your operational needs, you need a SQL database, right? Like Postgres, like MySQL, like SQL Server. I mean, you don't need it.
13:01 Speaker 03 I mean, you can use a MongoDB, you can use a NoSQL database. But you need an operational database, right? Let's just put it this way. And for the majority of workload today, people are using relational databases that that speaks SQL. And for a smaller part of the market, they use NoSQL databases. Which is more preference and user experience and whatnot. And then for an analytical system, uh, people use data warehouses, Teradata, Snowflake, BigQuery. And the interface to those databases is also SQL.
13:34 Daniel Whitenack? Yeah.
13:35 Speaker 03 And what you just started the podcast with is like, oh, I trained my model against the data that sits in the data lake or a data warehouse. Uh, and now I need to put it in production and my, you know, I have data quality, data consistency issues, my performance is not the same. A lot of that comes from the underlying data management. And if you like really peel the layers, it comes from the fact that you run this very same model on top of data and data management systems that are different.
14:04 Daniel Whitenack? Yeah.
14:04 Speaker 03 And one can argue that, well, they're different, there's they got their reasons for them to be different. But there's a more contrarian viewpoint here that is, we live in the world of clouds where things are abstracted away from you. And that gives an opportunity to build ideally a serverless interface that speaks SQL. And that gives you access to all your data and gives you access to all your data for reporting capabilities, for low latency capabilities, for operational workloads. And that would allow you to never leave your data universe as you go and move from one workload to another.
14:48 Speaker 03 And that can be huge, right? That can be huge. Because whatever data you trained, for example, your example, whatever data you use to train the model lives in that ocean of data and that data is easily accessible to you. And then you train the model. So now you need to convert new data that is incoming or, you know, marry new data to an old data and convert it into pixels, which is your app or a website or whatever. And that can be done right there off of the same data set you've been operating on. Which is certainly not the say the case today. Today you have a data lake, a data warehouse and a number of operational databases
15:28 Speaker 03 that can be integrated by a third piece of software like ETL tools or integration tools, which just generates a lot of complexity. And a lot of that can be simplified if you imagine a world of having a serverless SQL low latency API to all your data.
15:48 Speaker 03 That's the vision where we driving towards. And this is a multi-year, probably multi-decade. MySQL is 90 years old today. So it's going to be a multi-decade kind of life's work. But the workloads that we see emerging and the new workloads that are enabled by a system like this are real-time analytics and real-time decision support. When you need to kind of go back and look at the history of what was happening to make a real-time decision and do it at scale as well. So that is something that we see a lot in financial markets. You know, MySQL is a give or take $40 million run rate company with 70% growth. We just had an article on TechCrunch where revealed our numbers.
16:31 Speaker 03 And a good amount of that revenue is coming from financial markets. And if you think about it, like, that's what happens there, right? There's constant stream of information that's coming in modifying that data state that you have, and you need to make decisions about, you know, buy, sell. You need to make decision of, you know, in wealth management. You need to make decisions in portfolio management and trading. But also you need to make decisions in various systems that, for example, monitor, you know, something that's very, very large, right? So in Morgan Stanley, for example, it's a, it's a trading system. And we monitor this trading system, providing decision support to, oh, should we provide
17:16 Speaker 03 some sort of maintenance? Should we reroute our trades? You know, all of those things. That's what MySQL is used for today. And that's new. Like, we didn't have a system on the market that had those capabilities before.
17:38 Speaker 01 I'm Jared Santo, JS Party's producer and one of nine regular voices you'll hear on the show. We are a party-themed podcast, so fun is at the heart of every episode. One way we keep things fun is by mixing it up and trying new things. We play games like JS Jeopardy.
17:53 Speaker 04 This gives you access to an outer function scope from inside an inner function.
17:58 Speaker 01 Oh, I think that.
17:59 Speaker 05 Global scope?
18:02 Speaker 04 Incorrect.
18:03 Speaker 03 Yeah, I didn't think so.
18:05 Speaker 01 Debate hot topics like should websites work without JS?
18:08 Daniel Whitenack? I'm going to appeal to authority and read some quotes at this time. Okay.
18:14 Speaker 01 I've lost complete control of this panel. Go ahead, Ross.
18:17 Daniel Whitenack? The first quote, "No code is faster than code."
18:20 Speaker 01 Discuss and analyze the news.
18:22 Speaker 06 Yeah, this reminds me of when you're playing Pokémon and you have like, you know, an electric Pokémon versus a water Pokémon and you try like an attack.
18:30 Speaker 01 Share wisdom we've collected over the years.
18:33 Speaker 05 To be honest, a lot of what we rely on is pretty garbage.
18:41 Speaker 05 And like, I mean, I wrote some of it, so it's okay. Like, I can say this.
18:45 Speaker 01 Interview amazing devs like John Rezeg and Emilio Wattenberger and a whole lot more. Oh, and did I mention we record the show live? We do. You can be part of the hijinks each and every Thursday at changeodd.com/live. This is JS Party. Please listen to a recent episode that piques your interest and subscribe today. We'd love to have you with us.
19:12 Daniel Whitenack? I definitely liked where you're going in terms of describing the sort of single window to all of your data via the SQL interface. And I know that we talked a little bit, so we kind of touched on the AI and machine learning elements of this and how they fit in. You're talking about going on this journey to create the single window of, single interface to all of your data. How did AI and machine learning workloads start to cross your path at MemSQL and start to be something that you felt like needed to be part of the strategy of how you were building out the system?
19:50 Speaker 03 Yeah, this is a great question. When we did our analysis, we discovered that about 20 to 30% of all the workloads that MemSQL support have a, some sort of machine learning or AI angle to this.
20:06 Daniel Whitenack? Oh, wow.
20:07 Speaker 03 So this is a very large number. And when we looked at it, you know, we always wanted to have AI, like dedicated AI capabilities in the system. And we certainly use AI internally to make certain decisions around workload management, query optimization. But the fact that the modern workloads, and obviously people put modern workloads on MemSQL.
20:28 Speaker 03 Have a lot of AI and ML capabilities was eye opening to us.
20:34 Daniel Whitenack? Yeah, in those cases that you noticed, was it like people that were like you were saying, using MemSQL to do kind of large queries to prepare their training data for an AI model?
20:45 Speaker 03 Actually, both.
20:46 Daniel Whitenack? Both? Okay.
20:47 Speaker 03 And like two specific examples. Well, we have a great integration with Spark.
20:53 Daniel Whitenack? Uh huh.
20:53 Speaker 03 So we have a Spark, MemSQL Spark connector that allows, gives you very fast data exchange between MemSQL and Spark. You know, fast mean multi cluster to cluster. So like a multi-channel bus between the two. We noticed that people put all their data into MemSQL, grab it through Spark, store model somewhere else. So we don't take part in hosting models. So this is the first part. And what people like about MemSQL is that two-way path for data exchange between MemSQL and Spark. If you have something in Spark, you can persist it in MemSQL. You have something in MemSQL, you can pull it into Spark. And MemSQL is a world-class query processing engine. So you can send SQL query to it
21:37 Speaker 03 to do the kind of the first pass and slice and dice data before it gets fed into training algorithms. Which MemSQL itself doesn't support. It's just the backbone for that data. And the second use case that we started to see being pronounced is, well, people build apps on top of MemSQL. And those apps have, you know, models, evaluate models real time. And they need to, usually there's some sort of an SLA for an app either displaying this information to the end user, or the app is completely kind of back office and they just, you know, crunching data. And for that, they need to pull data from somewhere, run this data against a model.
22:21 Speaker 03 And based on the results that you see from their model, you know, do something. Typical examples, fraud.
22:29 Speaker 03 And we do in transaction fraud detection for some of the major banks. Where the SLA is forty milliseconds to make a decision if that particular transaction is fraudulent or not. And in order to make that decision, you need to go, you know, you have a model, that's model is already trained. Then you need to grab some data for that specific account, go back and look at the previous, you know, thousand transactions. Feed those transactions against the model. And then the model will tell you it's a fraudulent transaction or not. So MemSQL is supporting use cases like this.
23:05 Daniel Whitenack? Yeah, that's really interesting.
23:07 Speaker 03 Right? So again, like both sides of the spectrum, right? Both just providing basically a data lake or data warehouse capabilities. Put all your data in one place, let data scientists play with that data and use whatever data science tools, the tools du jour, right? Should it be Spark?
23:24 Daniel Whitenack? Most talks SQL.
23:25 Speaker 03 Yeah, yeah, yeah. Should it be Spark? Should it be Pandas? Should it be TensorFlow or PyTorch, whatever? We provide very, very fast data exchange to whatever frameworks you use. And the second one is, oh, I want to put my model into production. So I'm going to register that model somewhere in my, you know, either in Kubernetes, SageMaker, you know, there are tools for that now. And it's a rapidly evolving space. But it all starts with data anyway. So you need to have a data backbone and you need to have data management system with system of record capabilities. In order to provide, you know, uptime, low latency, all of those things. And where it's going is we're thinking to keep building world-class integrations.
24:09 Speaker 03 With systems that both data scientists use for training and engineers use for putting models into production. To enable that exchange from a push-button standpoint. Given you have a model, put that model somewhere, tell MemSQL about that model. And you'll be able to consume that model either from SQL, through user-defined functions. Or through an application, query the model, query the data, you know, and the application provides the glue.
24:37 Daniel Whitenack? Okay, so yeah, I was curious about that piece. So it sounds like right now the sort of workflow is you have an application like a Python application or whatever it is, you load your serialized model into memory. And then when it's time to fulfill a user request, then you make a SQL query against MemSQL, get the data you need, run it through your model and respond to the user. Is that about right?
25:03 Speaker 03 Yeah, that's how it works today. And where it's heading is this will still be a probably 50% of the use cases because certain things you still want to control and write very, very custom logic. But we want to make MemSQL aware of models that are stored in a particular repository and being able to through SQL to run data through those models and return results back into MemSQL.
25:28 Daniel Whitenack? Yeah, that's really interesting.
25:29 Speaker 03 Yeah, the reason that's useful is that sometimes you want to run that model against a very large volume of data. And so if your application row by row pulls data from a database, rounds it against the model, gets the results, potentially stores it back in the database, that is an extremely inefficient way. But what you can do is you can establish a similarly to Spark Connector, a multi-channel bus with optimized data format. You know, we're thinking Apache Arrow or something like this, where running a model against a billion records should be, you know, one or two second proposition.
26:11 Daniel Whitenack? Yeah, that's awesome. I'm thinking of like the use, like a facial recognition use case or something like that where like you may want to run like compare the embedded representation of this image against like, you know, thousands and thousands and thousands, or maybe even millions of records that you have in your database that are reference, you know, faces from your facial recognition or something like that. Am I following the right sort of path here?
26:40 Speaker 03 You are, and we have use cases like this where people store feature vectors in the database in a way people run this use case in MemSQL from the do it yourself kind of way. Meme SQL supports vector tensor operations.
26:55 Daniel Whitenack? Okay.
26:55 Speaker 03 As a built in. And obviously, if you, uh, facial recognition models, not all the time, but often is represented as, you know, kind of a tensor flow. DAG, I would say that evaluates and, and the individual nodes in the DAG are vector math. Right? They're not something that's spectacularly complex. It's a vector dot product, uh, also known, uh, as a scalar vector multiplication, right? So Meme SQL does that. And we have customers that in production do facial recognition over millions of faces. To enable things like, you know, somebody walks into a supermarket and they want to custom tailor the experience for that person.
27:39 Speaker 03 Or security systems in airports. And what happens is there's a camera, the camera looks at the next face. The feature, custom logic extracts a feature vector out of the new face. And then you run a query against Meme SQL that says, give me all the records where vector dot product of feature vectors stored in the database. Multiplied by the vector you just received is, uh, you know, between point nine and one.
28:07 Daniel Whitenack? Yeah.
28:07 Speaker 03 And that gives you all the similar faces. And because Meme SQL is a distributed system, even though it's, you know, it's a brute force way of doing it, there's no index. You just go literally run the dot product against, you know, millions of faces stored in the database. But because everything is so tightly optimized, you can still run this within, you know, fifty to a hundred milliseconds.
28:29 Daniel Whitenack? Yeah, that's crazy.
28:30 Speaker 03 And that's running in production, right? It's running in production. And like I said, for both kind of government, security. Use cases as well as, um, you know, things like walking into a grocery store. And the system suggests, oh, grapes that they usually buy are not in the system right now, but go buy this something else.
28:47 Daniel Whitenack? Gotcha. Yeah. So I was curious as we were talking and we were talking about, I guess that's a computer vision use case. And I'm thinking about like the types of data that are involved in machine learning and AI workloads. And we've got, you know, of course imagery and video. We've got a lot of natural language processing going on these days. And, you know, some of these types of data I've dealt with in SQL databases before. Of course, numbers and strings and that sort of thing. But I wouldn't typically think of like, oh, I'm going to store this image or video or like an audio file or something in, in a database. So are you thinking that in the longer term, the, a good workflow around this is that you're storing the sort of feature vectors
29:31 Daniel Whitenack? or embedded representations of maybe text or audio or maybe like spectrograms of audio via like the tensor built in or those sorts of things? Or are there other ways around that?
29:43 Speaker 03 This is a great question. To me, it's what it is now and what is it going to be as we go. And I will give you a very kind of product centric answer to this question. You know, like what would a product manager think? And they always start with a user. The user in this particular case is again, data scientist from the training standpoint and an engineer from building an app standpoint. I think today, data scientists and with the tools that the data scientists use, it's a lot more natural to store this data in a data lake, right? Basically in S3.
30:17 Daniel Whitenack? Yeah, just.
30:18 Speaker 03 It's bottomless, it's files, it's cheap and all the toolsets work out of the box. And the reason to put that data into a database is only when you get some sort of additional benefits to that. When you put structured data, the benefits are obvious. The aggregations, so it enables low latency access to that data and enables very fast aggregations and reporting. So you can slice and dice that data in the database before pulling the data out and use your custom tools to provide reporting. For unstructured data, the only benefits that I see are governance. Database can provide that unified access layer to all your data,
31:02 Speaker 03 but it doesn't give you any compute benefits over that, right? You get a kind of, kind of a wash. So that's the way we think about it right now as well as exploring. I think. What's gonna happen in the future? That databases, just like MemSQL, will give you an option to access that data that's stored in the data lake and in the file system through the database API, with the benefit of marrying that data and like really understanding metadata, potentially building a full text like index against that data. So you can marry that data with the rest of your enterprise data, which is usually relational.
31:41 Daniel Whitenack? Gotcha.
31:42 Speaker 03 But do not yank the direct access to the file system, because that's what data scientists do every day, and they would be confused.
31:53 Daniel Whitenack? Uh huh.
31:54 Speaker 03 If you remove that access pattern from them.
31:56 Daniel Whitenack? Yeah, I guess on that side of things, we kind of talked a lot about the operationalizing of models. On the training side, now we're kind of talking about access to files and all of those things. And you're saying you have the one interface or the integration with Spark. You know, for me, a lot of times I store everything in S3. Like you're saying, it's very natural for me. I just like say, I want this file and I'm gonna use it. But there's definitely issues that come up very quickly on that front, too. I know even like this morning, it's like trying to like deal with, you know, like 200 gigabytes of of audio data and like the I was just sitting around for a while and making coffee. It's not very productive or or fun to deal with those sorts of things.
32:42 Daniel Whitenack? When you say like, I guess on the training side of things, you have maybe people that are used to the Spark interface can do that. Are there other ways with MemSQL that like? Like if I want to access my audio files in S3, is there a way to do that with MemSQL outside of Spark? Are there other sorts of interfaces I can use?
33:02 Speaker 03 Not at the moment.
33:03 Daniel Whitenack? Okay.
33:04 Speaker 03 But I will share some of the thinking, right?
33:06 Daniel Whitenack? Yeah.
33:06 Speaker 03 So right now, there's a lot of technology we're building around.
33:12 Speaker 03 Just relational data and providing that single pane of glass window into all your relational data. That's where we're kind of the strongest. When we think about S3, we think how we can offload all the data that's not currently touched by the system into S3. We call this thing bottomless and making databases bottomless. Like, if you think about Postgres, Postgres is not bottomless, right? It's bound to the amount of hard drive that you run Postgres on. But we want to make it completely bottomless and very, very cheap. Uh, and S3 is, is, is probably one of the cheapest way to store data in the cloud. And we have things like MinIO that is one of the cheapest way to store data on premises.
33:54 Daniel Whitenack? Yeah.
33:55 Speaker 03 When specifically around that pattern that you described, right? They have an audio file and it's 200 gigabytes and it's a pain to go and transfer that file from one device to another. And it's a pain to download this from S3 to your local storage and all those things. So the thinking there is again is through integrations. If MemSQL is aware that here's the file in that particular format stored in S3 and then you want to somehow either bring computation to data or you want to, uh, access to a subset of that file. And only that you want to bring into your training environment, also either running on the cloud or somewhere else. So we want to enable those things. That's where it stops so far.
34:40 Daniel Whitenack? Yeah.
34:41 Speaker 03 That's where our thinking stops so far. We're certainly aware of the scenarios and we're aware of some of the pains that people go through. The place where we think MemSQL can add value is versioning. Because you oftentimes need to run and rerun experiments. And the model, it's not just the model, it's the model and the data that's been trained on. That's really the unit that is consistent. And if the data changed, the model might be rendered obsolete, might not be. So it just versioning makes a ton of sense from the ability to run experiments, verify experiments, share and exchange the models and data across data scientists.
35:22 Speaker 03 So I think that's where we can provide a nonlinear amount of value over time.
35:39 Speaker 01 Changelog News is the best way to keep up with the ever-changing world of software. We track, blog, and contextualize the coolest projects, the best practices, and the biggest stories each and every week. Make Changelog.com your daily destination, or hit the snooze button and subscribe to our weekly newsletter that hits inboxes on Sunday mornings. Join more than 15,000 enthusiastic readers. It'll cost you exactly $0, and you can subscribe right now at changelog.com/weekly.
36:27 Daniel Whitenack? All right, well, turning now a bit, I think, from the sort of AI and ML integrations, maybe to more sort of analytical workloads. I know that when we were talking before the show and in conversations leading up to the show, it sounds like that, you know, there's some pretty interesting things going on in terms of MemSQL being used during the COVID-19 pandemic. And of course, there's interesting, you know, tracing work going on and all of those things that I've heard about, but I haven't really heard about how some of those things are being enabled. So I'd be curious to hear a little bit more about that.
37:04 Speaker 03 Definitely. So let's step back for a second and think about what's different parts of the world and different companies and governments, what do they fundamentally want to accomplish as we go through the pandemic? The first one is, you know, simple, how do we stop the spread of the virus? And, okay, well, maybe we cannot really stop it, or let's say we put our actions and efforts to do that, but since it's spreading, it's a matter of fact. What else can we do and how we can drive our decisions based on data? What kind of decisions? Well, could be capacity planning for ventilators. All
37:43 Speaker 03 right, we know that, you know, there's an outbreak there, and we will likely have our healthcare system overrun, and we need to provide extra capacity to the healthcare system, but how much capacity? So all of those questions require answers, and the answers are in data. That's where data science comes in. And that's where just starting from collecting the data, putting it in one place, organizing the data and feeding this information to people who have the levers of power. Second one is like, who owns the data? We have obviously Apple and Google who own the data because they have a device.
38:24 Speaker 03 Every individual on this planet, not every, but most of them now on a smartphone. So you can tap into that stream of data and getting information about who is at which location at any point in time. And then marry that location with migration patterns and marry that location with like individual tracing. You know, given that we know that this person has COVID-19, you know, who are all the people that this person came across in the past two weeks? So we can go reach out to them and say, hey, you probably want to be tested. The second entity that has that data, obviously maybe government, but I don't know about that, but certainly telcos.
39:07 Speaker 03 Telcos have this information, maybe not as accurate because they don't have a GPS on the device. Well, actually they do GPS on the device, but they may not be able to tap into the GPS. But they can triangulate their location based on cell towers. So we're working with some of the largest telecommunications operators here in the outer space as well as around the world. And I think the one that's public is True Digital, one of the largest telcos in Southeast Asia. And we do both. We do the migration patterns where, you know, if you go back to like, you know, March, February timeframe, we already knew that there was an outbreak in China and there was an outbreak in Italy. And we already knew how bad that was.
39:50 Speaker 03 And looking at the flights from Italy and like tracing individuals that land and then starting to see this pattern of people getting sick emerge, you can start driving decisions off of this. And you can start putting policies in place that can stop the spread. You can start do capacity planning. You can start manufacture masks and ventilators and distribute them into places based on the patterns that we're observing. And so that's how data management solutions are helpful to companies that have the data. And also the insights that those systems generate are useful for people with the levers of power to drive policy and to drive decisions. Yeah, Google and Apple, well especially Google, has the technology, but you know,
40:35 Speaker 03 telcos don't. And that's where we partner and give them those abilities.
40:39 Daniel Whitenack? Yeah, it strikes me that, you know, the things you're discussing, there's definitely a lot of potential and value there. And earlier on in the episode about facial recognition and a lot of things that are possible there on a sort of large scale. And I think that as people are now in this pandemic and kind of layered on top of that, all of the climate that's in our country and around the world, around, you know, injustice and policing, there's a lot of people asking really good questions about actually data management and security and privacy. And I'm curious, you know, with you being in a position to have so many conversations with different types of entities around like how they view
41:23 Daniel Whitenack? data management, how that's changing as we think about these powerful applications of of kind of large scale analytics, but also the potential concern with privacy and tracking and all of those things. I'm just curious to get some of your thoughts on how large organizations are starting to view data management and security, maybe now a little bit different than than they might have in the past, given that all of the things that are going on in our in our world.
41:54 Speaker 03 Definitely. It's a multifaceted question. It starts with data management to highlight everything that's going on and the big problems that we face and big issues that we face as a nation. How can data management help here? And I think one of the answers to that of many, right? There's so many things that would go into solving this big issues that you raised. But one of those things where data management can actually help is with data sharing and data consumption, right? Imagine the police data was given by the government to the whole world in the easiest way from the consumption standpoint. And
42:37 Speaker 03 it's completely real time. So if you have an arrest and that arrest by regulation has to be a part of a public record, that is in the system in 10 seconds after that arrest happened. And so that information is just live, real time for everyone's consumption. And with our vision of a single pane of glass towards all your data and all workloads, we will be able to enable those things, enable anybody to log in into our cloud service and consume that data, assuming the provider is willing to publish that data. Imagine that climate change data is available to anybody in real time and it's live and it's easy to consume.
43:20 Speaker 03 So where we live today is a lot of data sets are public and a lot of data sets are public and there's regulation that forces them to be public, but they are published in a non-standard, obscure way.
43:33 Daniel Whitenack? Yeah, they're not discoverable.
43:35 Speaker 03 They're not discoverable, so to consume that data set is, it's a project. It's like going to a library or going to a court and like asking for permission and they will like bring this papers and put it on the table. I'm inspired by rewatching rather Spotlight where they got access to some sensitive data that had to by law be public and had to, they had to jump through hoops. But imagine all that data is discoverable. It is at your fingertips and that data is up to date. So you don't have to think about it like, oh, like I downloaded this from last month, what changed between last month and today? So it's just there. That can make a lot of things easier, more transparent and we'll be living in a better world.
44:18 Speaker 03 We need to think about the implications of that from the like, you know, what if bad guys had access to this data? But that's a policy question. That's not a data management question. I think data management should enable us to live in a world like this and the technology is already there.
44:35 Daniel Whitenack? Yeah, and I imagine that if you have this sort of single way to interact with data that's centered around SQL and people are familiar with that, they're able to use it. There's also, in addition to the sharing of data, there's sort of the sharing of methodologies that can happen. For example, even in our last episode that we recorded, we talked about some tooling that's out there around fairness and bias and other things. It's a little bit like you have to read a good amount of documentation. You have to figure out how to use these things. I wouldn't say it's like seamless and, you know, easily integrated into your workflow at this point. But I could imagine, for example, if a suite of tooling that is easily accessible via certain SQL workloads
45:21 Daniel Whitenack? that look for bias in your data on certain, you know, features or highlight certain things in your data set and all of those things. And whether you're using, like you say, whether you're using TensorFlow or PyTorch or Spark or whatever, you could potentially have access to those things in terms of people sharing their methodologies because things are centralized in terms of the SQL language. Do you see that? I'm wondering, you know, what's the MemSQL community like? I guess in terms of, you know, people working on projects built on top of MemSQL. What's that community like? And do they share certain things like that? Or certain things available that are maybe open source that are built on top of MemSQL that people can work on in a collaborative way?
46:06 Speaker 03 The community is on the forum.memsql.com. And then there's a community of mostly enterprise developers, actually, because that's has been our focus so far, that are sharing through MemSQL events and conferences. Where we're going is, you know, now that we've gotten here and we're opening up the platform more and more to the community, we're thinking a lot in terms of free and how we can make a lot of the things that got us here, you know, to got us to the 40 million run rate with 70% growth. How can we take some of that and open them up? And by opening up, providing certain set of features and capabilities to the world for free.
46:48 Speaker 03 So on our dime, you go in the cloud, you log in and there's this free tier of stuff that you can do. That's our current thinking so far and I'm actually going to be personally overseeing that effort here at MemSQL.
47:00 Daniel Whitenack? Yeah, that's really exciting. I'll be excited to kind of dig in and play around with those things. One other thing that I guess is COVID related and also related to our changing world is I guess people's just workflow and productivity during this time. I'm just curious with MemSQL growing so fast and obviously a lot changing, a lot happening. How has that been for MemSQL and how do you see kind of, I guess, tech work from home and productivity sort of stuff moving forward from your perspective as a CEO?
47:35 Speaker 03 Yeah, so first of all, we are in the uncharted territory, right? MemSQL wasn't the company that was born remote first, right? Even though we're global and we have office in San Francisco, Seattle, Lisbon, Kiev, Ukraine, India, I think it's Bangalore, and sales offices all over the place. There's still concentration in each location and usually a particular
48:01 Speaker 03 concentration of like a component that people work on within an individual location. We weren't impacted from our performance standpoint. It's like it's been one quarter of COVID. That was basically just finished our COVID quarter. We demonstrate tremendous results. We're very happy and excited about the future. And we obviously shifted all our workflows into working from home workflows. Now, the worry that I have, and I'm being paid to be paranoid, is that it works fine so far because we are tapping into the social capital that we've built over the years, right? And a quarter of COVID is we spending that social capitals and all this like social links
48:46 Speaker 03 are established between people and they fill them while working, you know, at a particular location and looking into people in the eye, to their friends and colleagues. So that's gone, right? So every meeting is a formal meeting, if you think about it. We're missing out on hallway conversations.
49:04 Daniel Whitenack? Yeah, I guess I hadn't thought about it that way, but it's true.
49:07 Speaker 03 Yeah. We're missing out on hallway conversations. We're missing out on, you know, uh, grabbing coffee together and having this like nice positive experiences, brainstorming while walking towards a, a, a nice coffee shop and, and grabbing a latte. So I want those things to be back.
49:28 Speaker 03 So hopefully this will, will happen relatively soon and we'll have a dent in the social capital that we've built and then we'll kind of fill up that dent by getting back together.
49:39 Daniel Whitenack? Yeah.
49:40 Speaker 03 So that's my hope. Obviously we can't control that. The situation controls us a little bit.
49:45 Daniel Whitenack? Yeah, it's interesting. I mean, so I've been working remote, previous to COVID, I've been working remote for I think maybe about three or four years now. And I definitely get what you're saying. I, I, I've had to intentionally over time, like develop relationships with local like data scientists or technical people that are, you know, maybe not, so they're not working at the same organization that I am, but it's a chance for me to like get together with those people and just talk about things. Cause sometimes I wonder, just sitting at my computer, I brainstorm a lot of things. And sometimes I wonder if I'm crazy because I'm just, I'm not talking about those things to anyone, except in when I'm like presenting them to my supervisor, I'm presenting
50:27 Daniel Whitenack? them to a group and I'm supposed to, you know, sound like I know what I'm talking about, hopefully a little bit. So that, yeah, it's not that sort of informal, um, environment. And that's very interesting observation. I hope that some of that can come back.
50:41 Speaker 03 Definitely.
50:42 Daniel Whitenack? Yeah. As we wrap up here, uh, love to give you a chance to just let people know. Obviously there's, uh, memsql.com. We'll have the links in the show notes, but as like a data scientist or AI sort of person, are there ways that people can kind of play around with memsql and get a little hands on and see what it feels like and how to do certain things? Where would you recommend that they start getting on boarded?
51:09 Speaker 03 Definitely. So if you want free forever, we have our software and we give our software to up to four servers, like I said, it's a cluster software to install whenever you want and run forever. So we call this our software free tier. It grew three times over the past year from the active user standpoint. It's basically one of the best column stores on the planet. So data is highly compressed, stored on disk, very fast reporting, everything is updatable, transactional system of record. And so where other companies like, I don't know, that run on premises, you know, the vertices, the green plums, they want to charge you for that, you know, you get it free and you can put billions and billions of data
51:52 Speaker 03 points in the system and get very, very fast SQL response from it. In the cloud, our free tier is time based. So I encourage people to log in and you know, you can play around with a system that would allow you to not use any software and consume everything as a service. But because we're running it on our infrastructure, we're limiting access to free for a period of time. We'll be announcing more changes there. We'll give the system for you for free forever for a limited usage, but that's hasn't come out yet. So that's something we're working on. So that would be probably the best places to start. And of course, go to forum.memsql.com to learn about the system.
52:32 Daniel Whitenack? Awesome. Yeah, we'll have those links in the show notes. Really appreciate you chatting about everything today. I think our listeners will really enjoy the content and hopefully check some of these things out. And yeah, thank you so much for joining us and hope to have one of those hallway chats with you some point when when things are actually opened up.
52:52 Speaker 03 Well, if you're in Silicon Valley or I'm there, I will make sure to paint you and we'll hopefully make that happen.
52:58 Daniel Whitenack? Yeah, definitely. Yeah, thank you so much.
53:00 Speaker 03 Bye, Daniel.
53:01 Daniel Whitenack? Bye.
53:05 Speaker 01 Thank you for listening to this episode of Practical AI. People ask us all the time. They say, hey, how can I support your work? One easy way is to leave a five star review on Apple Podcasts. Tell folks why you listen and why they should too. It only takes about 30 seconds and believe it or not, those ratings and reviews really do help us rank higher in AI-related search results. Practical AI is hosted by Daniel White Neck and Chris Benson. It's produced by Jared Santo, that's me. And our music is brought to you by the one and only Breakmaster Cylinder. We are sponsored by amazing people at companies who get it. Thanks again to Facilit, Linode, and Rollbar. Did you know we have a master feat of all Change Log podcasts? We do. It's your one stop shop for everything we produce.
53:47 Speaker 01 If you like this show, you'll love the Change Log, Brain Science, and GoTime. Check it out at changelog.com/master or search for Change Log Master in your favorite podcast app. You'll find us. That's it for now. We'll talk to you again next week.
Previous Roles to play in the AI dev workflow Next The ins and outs of open source for AI Page updated October 26, 2026 · transcript and summary are machine-generated