1:00um, Hello, hello. Can you hear me? Sweet. Um, welcome, everybody, as, uh, folks are filing in. We'll just give them a minute before we get going here. Um, we are here to talk about flights, which everyone at MotherDuck is super excited about.
1:23Um, I'll let Du introduce himself in a moment, but I know he and the team have been working super hard on it. Um, I've been creating flights like crazy, and it's just been such a cool feature to dig into. Um, so we will get started. We're gonna spend most of our time today actually demoing, uh, after a short just, like, orientation on what flights is and why we built it. Um, the,
1:47as we get people coming in, just pop into the comments and share what your favorite DuckDB function is. And if you don't have a favorite, you can. Uh, jump into the docs real quick and find one. Favorite DuckDB function, maybe where you're calling in from, and we will get rolling here in just a minute.
2:09Just a double check. Everyone can hear me now, right? Du, you can hear me? Okay, great. I saw a comment earlier. Maybe we're having some some AV issues. Cool. Um, so I think we'll actually just get going here in a moment. Uh, just quick introduction. My name is Garrett. I work on the product marketing and dev rel team. Du, do you want to introduce yourself real quick? Yeah, for sure. Hey, everyone. Uh, I'm Du.
2:34I'm the product manager who works in flights for MotherDuck. Um, so yeah, great to meet everybody. Shout out, Greg. read_csv_auto is a fave around here, for sure. Real DuckDB enjoyers. Love that one. Um, cool. Welcome, everyone. Uh, appreciate you spending your time with us. Um, just real quick. I'm gonna just share a couple quick slides, like overview stuff.
3:00Again, we're gonna spend most of our time on a demo here. Just housekeeping. Um, please pop your questions in the chat as as we go through. We're gonna take those as we go, keep it nice and interactive, and anything we don't get to we'll get to at the end. So please share your questions as we go, and we might stop and just, uh, follow those wherever they lead us. So I'm gonna go ahead and
3:23present here. Actually, Jerel, you might have to. Oh, here we go. Thank you. Appreciate that. Um, just to double check, you can still see. Yeah, cool. Um, yeah, so we're gonna talk about flights. Flights is our agent-native data pipeline feature in MotherDuck. We're gonna talk about exactly what that means.
3:48Um, it's a really broad feature set that that we have now. And, uh, so there's all kinds of incredible things you can do with it. Um, but a couple core pieces that you should think about when we're talking about flights. Um, we built it to be, and I'm using this term here, it sounds like a little bit of a fluffy thing, but agent-native.
4:10Um, and so that means that the first class interface for building flights with MotherDuck is the MCP server. We'll talk a little bit about the other interfaces that are available to users. There's SQL table functions and a lightweight, uh, flights UI that we'll show. Um, but the whole goal was to be able to
4:30have a user get data from, you know, virtually any source that can be accessed with Python. Um, you know, run arbitrary Python code if they want on transforms and enrichments of data, land in MotherDuck, and then build that into a dive, which if you don't know are the other end of um, of MotherDuck with, uh, sort of agent-native data apps that you can build through our MCP server as well.
4:56Flights is super cool because it is essentially just a general purpose Python runtime that is tightly integrated with MotherDuck databases. So anything you can pip install, uh, you can put into a flight, which, you know, just let your mind go wild on what that means for a moment, but you can obviously ingest data through libraries like DLT that we will talk about a little bit more detail, transform it, enrich it, and do a whole lot more.
5:19Um, you know, I think about Python as like the close cousin of SQL when it comes to data. And if you want to fight about Python superiority for all types of other things, we could do that later. Um, the coolest thing, as I mentioned, is just what this has been unlocking for folks through the MCP server.
5:38So that single interface will help you go from adjusting data all the way to transforming it, normalizing it, and then visualizing it, of course. Um, just before we go much further, I just want to provide this as a resource. Our documentation team has built a, uh, a hell of a lot of examples that you can access through our documentation site cookbook,
6:01which you can find at motherduck.com slash docs slash cookbook, and it has some really cool features like the ability to just copy a really well written prompt for generating flights that extract and transform data from common sources in a way that is legible to the agent. And those are, you know, essentially one-shot implementations with a little bit of back and forth.
6:25So please check out the cookbook. Our team has put a lot of time into that. We hope it's helpful. So just quickly on use cases for flights, we're going to demonstrate some of these shortly. Um, many of our early users are using flights to ingest and transform data.
6:41So essentially, you know, being a general purpose Python runtime, you can use DLT, which is an extremely popular Python library for ingesting data. You can also use DuckDB community extensions. So if you're not already a MotherDuck user, DuckDB has a vast array of community extensions. Um, not all of them are supported on the MotherDuck server side,
7:05but with flights, you can install basically anything that you would be able to install through through DuckDB because you can access it through the Python client. You can, of course, do other things that Python enables you to do, like call APIs directly and use Python specific data transformation libraries or data frames libraries, rather like pandas, polars and so forth.
7:28You can also orchestrate. And one of the coolest examples is we've seen folks building flights to orchestrate other flights. Talk a bit more about this in a second, but flights come with a really handy interface, which is the SQL table functions. And one of those functions is MD run flight, which, as you might guess, runs a flight of your choosing.
7:54And so you can build chain executions of SQL. Um, we have an example of this in the cookbook, actually, of building a DAG using SQLglot, which is a SQL transpilation library. And, you know, that orchestration unlocks a whole lot of downstream use cases as well. And then, of course, you can also enrich and automate pipelines with flights.
8:17So in the middle of the flight, you could call an LLM to do things like classification, tagging, document extraction, summaries, and then any any external process as well. We've seen folks building, you know, report creation tools, slack bots, scheduled briefs, any sort of external notification that has a service that you can ping
8:40through a Python process. And again, I just want to emphasize anything you can pip and solve, you can run in flight. This is what it looks like and just how to think about at the high level kind of the the use cases. So existing in MotherDuck before and still to this day is Ducklings, which are our
9:03SQL compute instances built on DuckDB that access your databases directly with SQL. And now we just have a new compute paradigm that lives alongside those Ducklings through the Python runtime that is included with flights. So as a user you can create and schedule flights and then, you know, to manipulate and to to read or write data from
9:27MotherDuck databases, the flight attaches using the Python DuckDB client. MD colon is kind of our shorthand for connecting a DuckDB client to MotherDuck if you're not familiar. And then of course on the other end, flights can also read and access external systems as well. So if you wanted to do things that, you know, I think reverse ETL was kind of a term that was in vogue for a while for data activation as well.
9:51Like if you want to take data from a database and use it to hydrate a downstream system like a CRM or anything like that, you can do that with a flight also. And as I mentioned earlier, there are three main interfaces to using flights today. The first and most commonly used one that we've seen folks adopting is directly through our MCP server tool.
10:16So if you enable the MotherDuck MCP server, your agent, which is any MCP compatible client, will get a guide on how to create flights through the server. And you can of course do anything that you would be able to do with the flight through that endpoint. Create them, manage them, schedule them, edit them, and so forth, all in a natural language interface.
10:41I mentioned the SQL table functions earlier as well. So any MotherDuck connected SQL client can also manage flights with our SQL table functions, like things like run flight, delete flight, schedule, and so forth. And then we will make sure to demo this here in a moment, but we also have a lightweight flights UI.
11:03If you'd like to just copy and paste in some Python code or write it yourself, the flights UI also provides a visual interface for scheduling and configurations. It also is, you know, obviously within MotherDuck, and so you can manage secrets on the MotherDuck side as well through your organization.
11:28Cool. I have not been paying attention to the chat really quickly. I think most of the questions will come in the demo, which we will pop on over to now. All right here. And yeah, just a reminder, as due as demoing, please, yeah, put your questions in there. I will field them, and we will make sure that we get those answered. And I'll try not to interrupt you for the first minute or two, but
11:52everything's fair game after that. All right, perfect. Thank you, Garrett. For those of you who are just joining, just a quick re-intro. So I'm the PM for flights here at MotherDuck. So, all right, so I think, so today I'm actually going to show you how to bring external data into MotherDuck and turn it into a live, you know, interactive dashboard entirely through the natural language.
12:16And as I mentioned, I'm a flights PM, so no pun intended, but I'm actually going to be ingesting SFO's flight operations data today. So let me go ahead and share my screen. One second. All right, I think this should do. All right, can folks start seeing my screen?
12:37Yep, I can see it. Great, awesome. So let me turn over to the source data first. So quick context on this data. It's a public data set from SF's Open Data Portal. And, you know, as you can see, there's a bunch of fields, including flight times, arrivals and departures, airlines, runways, etc. So it's really rich enough to do something really interesting with.
13:00And what I'm going to show you today is something that I actually ran earlier this morning. It's like a full experience end-to-end. From bringing the real external data set, loading it into MotherDuck, and then turning it into like an actual live dashboard. And again, that's all through just natural language. So let's walk through it here. So, you know, Claude chat, you know, this is where it all started. I typed one prompt here.
13:24I told Claude that I wanted to pull SFO's flight operations data from the Open Data Portal and filter only to 2026 data and load it into MotherDuck. So that's it. No codes. There's no, like, you know, schema definitions. There's no API docs that I'm giving. It's just, it's really all just that. And you can see that, you know, Claude went off and it started calling the MCP server tools that Garrett just mentioned, right? So it's
13:49reading the guide and inspecting the schema and figuring out the pagination and writes the ingestion script. And it uses the SQL table functions to manage flights such as, you know, creating it, listing, and deleting or cancelling if needed. So let me turn over to our app UI here. So this is what it built. We have the SFO flight ops ingest here and
14:14this is the flight that just already ran a few minutes ago. And let me just give you a quick overview of what we're doing here. So, you know, these are essentially the logs from the run. You can see it paginating through the Socrata API in real time and it's fetching the rows and batches. So it's not a black box. Everything is very visible here. And
14:35this is the actual Python script that Claude wrote. It's all just plain Python. There's nothing proprietary about it. And also, if you can actually write a Pandascript, you can write a flight. The other two tabs here are just like dependencies, you know, flight handles, like installing these, like, you know, in the container automatically. So normally, you would not really have to go into those two other tabs. Let's look at the versions. So every time I change the code,
14:58you know, every editor's version. So you can always, you know, take a look at it and roll back if needed. So this is the version that I'm using right now. So this is the version that I'm using right now. So this is the version that I'm using right now. You know, every editor's version. So you can always, you know, take a look at it and roll back if needed. And here's actually, like, a really nice touch that I love. You know, the run actually
15:23renders both both the UTC and your local time zone. So it's a very small thing, but I think it, like, really matters, like, when you're collaborating across teams in multiple regions. So let's see. So what actually happened, you know, when this ran? So let's go back to Claude. Created the flight. It pulled,
15:46you know, 147,000 rows, only because we limited to just 2026, but there's a lot more data, and all of this is loaded into MotherDuck directly, and as you could see, it took roughly 48 seconds. It's really just, like, fractions of a cent. And you can see that it gave me, like, a full summary, like, as soon as it was done, right? So the table is called, you know, SFO data main
16:12dot flight ops 2026. It's got 23 columns that we saw earlier, and it's even suggesting some kind of a sample query that I should run to get started. So now, you know, the data is in MotherDuck, and, you know, I actually want to understand it. You know, what did we do with this data? So I'm asking Claude to build a dive on top of it. And let's jump over to the UI again. Actually, let's just scroll down here.
16:38So we have this MCP viewer tool, which allows you to actually look at the dive without, you know, hopping over to our UI, so you know, it's giving me, you know, a cool visualization that, you know, analyzes the data by looking at the hour of day, so you can, you know, you can kind of see the activities basically zero from, you know, 2 to 5 a. m. It ramps up a little bit harder at 6. You know, there's top airlines and
17:03runway utilization, and you don't even have to leave this chat. You can just keep iterating on it, and then it'll save new versions on the same file. Let's see. I think the last thing that I want to mention here is, you know, what I care about is, you know, the freshness of the data, right?
17:23So I'm going to ask Claude to put this on a daily schedule so that, you know, the data always stays fresh. And not only that, you could also do this on the UI itself by going into edit and for folks who are not super familiar with, like, you know, writing Cron, you can just use a natural language here again, like what I did here. I could say every Tuesday at 7 a. m.
17:47Pacific, and it updates automatically. So again, you know, it's you know, and then the dive will obviously always kind of reflect the latest data as well. So there's like no pipeline, you know, it's all just prompt. So that's the full end-to-end flow. I will take a pause there. But, you know, as you can see,
18:10we've got the entire external data and got it transformed and scheduled and visualized in a matter of like five, six minutes, really. So yeah, and again, just using Flights data to demo Flights was very, very, very much intentional. So I'll stop sharing my screen and turn it over to Garrett and the audience for any questions. Actually, could you pull it back up? There's some follow-up stuff. Sure.
18:34I want to make sure we see. And just while you're doing that, one softball question. I'm at a hard one immediately off the bat. Let's see if I can promote these. Maybe I'm not able to. Anyways, Sandeep asks, how are Flights different from dives? Adam has already answered. Thank you, Adam.
18:58So Flights are the data pipeline side of the house, right? Getting data into MotherDuck, transforming data that exists in MotherDuck, as well as pipelines out of MotherDuck databases to other systems. Dives, if you're not familiar, are essentially data apps that live inside MotherDuck. You can create classical BI experiences. You can create new age,
19:21you know, very interactive things. We're not focused on dives so much today, but I encourage you to check that out. There are actually embedded versions on our website that you can see. And since dives are just React and SQL, you can create virtually any interface that you would want.
19:40One thing that I'm going to nitpick you on, do, because I think it's really important, is flight secrets. Can you just explain a little bit, like how I see flight secrets here in the UI. Could you just explain a little bit how it works? Sure. Let me point to the UI as well. All right. So, good call out. So, when you saw the config tab earlier,
20:04that's essentially where the secrets live, right? So, you reference them by name in your script, like, you know, OS, like, environment, like, mother. token, and flies inject them at the runtime as environment variables. The key thing is. . . Oh, you see the secret name, so your credentials are, they're secure, even though the script is fully visible.
20:30Sorry, Du, I think you jumped out for a second, right while you were saying the important part. I think you cut out. Sorry. Okay, let me just kind of repeat the entire thing. So, as I've mentioned, you know, when you go into flights, let's actually cancel out of this, you know, we have the config tab here, right? There's nothing in here, but you, you know, especially because we are bringing
20:53just the public data in here, but essentially, that's where the secrets would live, right? And then you reference them by, like, name in your script, you would call them something like an OS, like, environment, MOTHERDUCK_TOKEN, and then flights would just kind of inject them at runtime as environment variables. And I think the most important thing here is that your actual secret
21:13value is never exposed, never in the logs or the code or the UI, you would really just see the secret name. So, you know, that being said, the credentials are, they're going to stay secure, even though the script itself is fully visible here. Yep.
21:33Very cool. Stephen asks, hello, Stephen, thank you for joining us. Do you have a template for how to run DLT on flights or other common data engineering workloads? Yes, absolutely. Please pop on over to our cookbook in the documentation. You just head to motherduck.com
21:53slash docs, and I believe it's slash docs slash cookbook. You'll see examples for several other MotherDuck features, including flights. And DLT is definitely one of those examples. Okay, hold on, we got like a big backlog of questions here that I want to get to.
22:14And then selfishly, I also have a cool demo that we might do, but we'll see. Brian asked about the flight SQL function. Can you show the flight SQL functions a bit more? Yes, I'm going to present here for a second just to show our docs. Okay, let's see, just pop that up real quick.
22:49Jerel, if you're able to help, my screen is stuck in purgatory. Woo, there we go. Thank you. So this is our documentation page as part of the reference for the SQL table functions that drive flights. I mentioned this earlier in the beginning, but these are essentially what the MCP server
23:12is calling when it is creating flights, updating flights, triggering them. But you can also call these yourself through any DuckDB client that connects to MotherDuck. This is what the syntax looks like. And, you know, this is just a select statement in SQL that holds all of the
23:32parameters for your flight itself. So in this case, like this is a very simplified version, we have the name of our flight, our source code, this will contain our entire Python script, the cron parameters, scheduling requirements, config, our access token that we're using. This is just the token name. This isn't plain text token or anything like that.
23:54It's a token name that already is managed in MotherDuck and the name of any flight secrets. And so that is sort of the syntax that we're working with here. And you can see all the parameters. So I encourage you to check out the flights documentation to see all those functions. There's a large list, right? We can do everything from creating, updating, deleting,
24:18listing, and then triggering, of course, with MD run flight, as well as canceling runs and getting flight logs even. So you could create this entire flow that Du just showed in the MCP server context in the Claude Web interface. You could do this with a terminal based agent without MCP at
24:39all, just using the MotherDuck extension on the DuckDB CLI. Thank you, Logan, for popping that cookbook channel into the chat. Appreciate you. Yeah, I guess, yeah, just to do what, you know, people said earlier, when Claude was calling flight tools in the chat, that's literally what was happening under the hood. It's just calling the SQL functions that Garrett was going over.
25:04So essentially, the MCP layer is like just in that it's a just a natural language interface that we're using on top of SQL that was already there. Yeah. Good question. Madison asks, how do you handle debugging governance if everything is prompt
25:23based? I'll intuit a little bit of the question here, and maybe this won't strike to the heart of it, but, you know, everything that we are actually building and running is just just Python code, like it's totally deterministic. We're definitely creating the initial flight in that
25:42Claude flow, as you mentioned, or as Du showed, but there's no like there's no LM involved in actually what gets run. The. So I hope that speaks to the heart of it a little bit, but if not, happy to get back to it. But yeah, you can totally really just also write Python
26:06and, you know, and create flights via that table function or you can write it directly into the UI as well. Yeah. And I think, you know, there's just to add to what Garrett mentioned here, I think there's a couple of things that are working in, you know, like more technical folks favor here. So we have the logs, right? Everything that runs in flights is
26:28fully logged and it's visible in the UI. So if anything breaks, you can see exactly where and why. And the second thing that I briefly hovered over was the versions, right? So every time the script changes, we store a new version. So you can always see that diff on what changed and, you know, decide to roll back if needed. And and I think you've also mentioned the governance side,
26:52right? We have so flights are tied to your mother that account and there's some access controls underneath it, right? So, you know, you would essentially know who created what, when it ran and what it touched. And yeah. And, you know, frankly speaking, you know, sometimes Claude does help a lot with like debugging if needed. So if a flight fails,
27:13you could, you know, what I typically do is I just, you know, copy and paste the logs back into the chat and ask to fix it. So there's multiple ways to do that as well. We've got some folks asking about source control. And I think this is, you know,
27:34we mean get probably. So Greg asks, what is the preferred way to manage and collaborate on code put into the flights? He followed this up. He means flights, not dives. And then someone else, I think, asked a similar question that might have been get specific. Do I don't know if you
27:58want to talk about directly or we can talk about kind of the some of the other paradigms? Yeah, for sure. So right now, you know, versioning is managed natively inside the MotherDuck, but every editor is stored as a new version. So you do get the full history of changes. But getting integration is actually something that we've been we are actively thinking about right
28:19now. And, you know, if that's important to your workflow, please let us know. We're happy to chat one on one in kind of like, you know, in a design partnership slash like a feedback session. So, yeah, that kind of feedback always helps to shape the roadmap for us. So please send it our way.
28:39Yeah, I'll say from for our part as well. You know, I can feel how important this is to folks. And I don't want to share like this is not a workaround by any means, but the SQL functions themselves enable basically like a version control and collaboration workflow like we have built
29:01internally for for dives, which operate in a similar way. You can basically like create a CI pipeline that run that runs on merge the update flights function. And so you can version control that way and use a service account in order to, you know, to manage creation and update
29:25across the team. So it's a little, you know, it's not quite like maybe what you might call native integration flow that feels quite familiar when you have that source control and GitHub and then CI, you know, GitHub Action, for example, to run the update flight SQL function. Yeah, I think Nick's I think Nick's kind of on to this basically.
29:51Yeah, exactly, Greg, like deploy, deploy the dive or the flight from GitHub with actions. And then the action runs the runs the SQL table function to update and then run the resulting
30:05flight. Definitely. OK. Did you do so many questions? Thank you all for this. Appreciate it. So Sean asks. Dumb question, but how are these flights shared? Can you create a view or access
30:29permission structure? Can I tie into Azure groups? Yeah, I do. I want you to see this one for you. Yeah, no, it's not a dumb question. Go ahead. No, I was just about to let you know how little I do in the Microsoft ecosystem. I said Azure instead of Azure.
30:54No, not a dumb question at all. It's actually like really important. Right. So right now, today, you know, flights are owned by the creator, you know, but, you know, essentially the tables they produce, you can share them through shares. And, you know, just
31:14a quick preview. We are in the middle of, you know, working actively on our back right now as well. So, you know, down the road, that'll help really kind of control access to it. And we I guess like the so I guess like does that answer your question first? And then I think the second
31:35part of the question was, like, can I tie into Azure groups right now? No, but, you know, we can consider that in our roadmap. And if that's a hard requirement for teams, definitely worth flagging it. So, yeah, please let us know. Cool. Thank you. Thank you.
31:56Oma asked a little bit ago, how does flights handle a medallion architecture? I actually think these are two different concepts, really. I mean, you know, the medallion architecture is just how you've built your infrastructure under the data warehouse. And so, you know, you can, for example, if you have if you're just running plain old S3
32:19as a landing zone, you could curate and ingest data into MotherDuck using a flight. Of course, you can read directly from S3 with MotherDuck. But for whatever reason, if you don't want to do that, if you want or if you want to apply some Python driven transformations to that data as you ingest it, you can connect with a flight to do that. So
32:41I actually think these are quite complimentary and pretty flexible. Again, anything you can do with Python, really, or the DuckDB client in Python, you can do with the flight. OK, this is interesting. Yeah, Brian says, feel free to ignore if not appropriate. Stepping a level up, have you seen non-technical people take to using these chat-based tools?
33:08So I think by these chat-based tools, you're also referring to dives, which was sort of our first dashboarding a data app side of a very similar user experience. Definitely between dives and flights, we have seen this type of work become accessible to much less technical people. I think flights itself is quite early. And as you can see,
33:33we're actually looking at Python code. There is code being written rather than just a dashboard being generated. But there are folks on our team internally that have been using it to get data from finance systems or HubSpot into MotherDuck, which is super promising because they would not have thought of themselves as data engineers. But those patterns are so well
33:55defined now. Thousands or hundreds of thousands or millions of people have gotten data from CRMs into a warehouse that's been around for a long, long time. And those APIs are quite stable. And so libraries like DLT enable us to point flights at those things and effectively create pipelines. So I think we definitely will see this become accessible to more people.
34:20And that does not get us off the hook for good data engineering practices and ensuring the warehouse isn't turning into everybody's landing zone. But with the right type of coordination, it is certainly accessible. Cool. We have one other question about pricing, which is great.
34:44So this is all our pricing page, by the way. I apologize. I should have put that in the slide. Flights run at 60 cents per hour. Today, it is just a single size of compute instance, and they are billed by the second in the same way that your ducklings are. So your bill for a flight
35:08will be the time that the flight runs down at the second, plus the time that a duckling is up to ingest data or to read data to the flight or however you're using it. So those are the two components here.
35:28There's another question I think a few people are asking about, like, resource sizing, right? Like, what can you actually do here? Du, I think that's a good one for you to tackle. For sure. So for, I think the question actually came from Steve, right? So for compute, today, we have one container size, and it's designed for, you know, typical data pipeline workloads. So
35:52it's like 32 gigabyte memory workloads, and, you know, very long-running, like, heavy jobs are, like, really, like, at the edge of, like, what we would recommend for the current preview. And, you know, we're actively working on expanding the compute tier, so if you have a specific heavy workload in mind, we'd definitely love to hear it and reflect it in the work that we do in the future.
36:19Oh, another hard one. Sorry, Du, is there any keep going? Thanks for joining us, Lucy. Hello. Lucy asks, how do you use your in-house Python library? This sort of seems to assume that we are all using open source, right? I'm sure you guys have cooked some cool things. Yeah, I don't know the answer to this, Du, maybe you do.
36:46It's a good question. So you can actually bring your own, like, private Python libraries into flight, right? You would just reference them in your requirements, or even, like, just package them alongside your scripts. So say, like, you've got, like, internal tooling, you know, it's not locked out. So that said, I think, you know, the details of how to set that up are,
37:11like, actually worth a follow-up conversation outside of this call. So definitely reach out, and you can walk through your specific setup. Yeah. Yeah, we know where to find you. There's a follow-up for you, Du.
37:29Um, yeah, similar question. Steven has, like, the, uh, anything you can pip install, like, on the open source, or on the pip side, like, anything you can pip install, you can run in a flight. It sounds like we have a little work to do documenting the use case for custom Python libraries or private libraries, so we'll definitely take that as feedback. I'd appreciate that, that question.
37:54Um, Ken asks, and I'm going to read this all in real time, sorry if this is too basic. Okay, ignore my path is to ingest as I print info to a PDF, and I have a customized scan to grab the fields I need that writes to Excel, and then writes to the database. Is there a better way? Oh, gosh. Almost certainly there's a better way,
38:18um, depending on, you know, how familiar you are with some of these tools. Like, there are great, um, uh, AI document parsing products nowadays. Like, um, Tensor Lake is one of them. Reducto is another. There are several. Extend, even. These are commercial products. I haven't tried
38:37the open source side of the house, but, like, you know, OCR and PDF document processing broadly, PDF document processing broadly, there is a lot to use in the ecosystem. Um, so Byte, for example, could call out to one of those services and process
38:57PDFs, um, as it comes in. Sorry, Du, you're looking confused. I'm not, sorry, I was reading a chat. Oh, okay. Yeah, so, Ken, definitely possible. I think there's probably a little research for you to do for your use case. Are they all the same format of PDF or are they, um, you know,
39:20are they, uh, totally unstructured and distinct? But, um, yeah, there, there are a lot of great libraries out there and a lot of great tools. I'm actually going to show you just as an example, um, how, and I was, I don't mean to hijack, but it'll be really quick. Like, but I just wanted to show an example of something I've done in a similar vein. I'm going to represent
39:44a different screen here. If Jerel, you could promote me. Cool. So I made this app that looks at the snowpack in Washington state, and it uses to all sources of data. Apologies for the jumping around. This is like radar, a radar
40:06overlay of Snowmelt. And we've got some satellite imagery as well. And this imagery is not like a SQL friendly task. And so instead, and I'm just gonna switch my screen real quick. Instead
40:25of using like trying to force that through a different route, I'm just going to show you what I built with a flight instead. So I have a couple flights that use modal, which is an AI infrastructure
40:45platform to do the image processing. So I'm importing it here, I have a secret for a modal account that I managed in MotherDuck that gets injected at runtime to do the image processing itself. So I've got actually three flights running that app, there's some forecasting I'm doing that needed some of that AI infrastructure power, as well as just bringing in, I'm ingesting data from
41:10public API, as well as, you know, importing that radar data. And so that's an example of like, a totally external task like image processing that I'm orchestrating through through the flight itself. So apologies for the sidebar that makes me feel happy to demo. Cool. It is in my private GitHub,
41:34and I have not cleaned it up. So give me a moment. And maybe I'll share and show and tell in the MotherDuck community. That's my plug to join us in the slack community. Because that's, that's where I put it. Cool, I'm just gonna, it seems like there's a lively discussion about OCR. So
41:56that's fun to see. Yeah, Brian, I forgot about that blog. That's a good example. Yeah, the folks at tensor like, we're, we're generous enough to write that for us. And I know they've built a lot in the in the meantime. So definitely check out that blog. For example, I think there's even a
42:12Google collab notebook, if I remember correctly in there. Yeah, I, I think we demoed here, like, do showed really, what's probably going to be a very common example of folks ingesting data from from public endpoints or using DLT. But yeah, I do just want to underscore that anything that you can do
42:35with with pip or in your private Python, you can, you can do here as well. Cool. I'm just going to surf this real quick for other questions. And that's some of the OC, OCR and document processing
42:52discussion. Can you get a static IP for flights? Oh, interesting. Real deep. Okay, dude, sorry, I have another one for you. Nick asks, Can you get a static IP for flights to whitelist? I was
43:15overlapping first to whitelist for whitelisting connectivity to source database? Great question. So first, as of today, flights actually did not have static IP support just yet. Again, you know, similarly, we're listening to, you know, hardware requirements for folks. So, you
43:38know, likely for like firewall whitelisting. So that's a good feedback. And, you know, we can always reach back out or have conversations about it as a follow up. Cool. Just shameless plug for our MotherDuck Slack community. If you just search MotherDuck
44:01Slack on on Google, you will find it but if you do not, it is mother. community. slack. com. If you have questions that we didn't get to hear or that you wanted to chat with folks more about, we don't want to lose everybody. We can continue the conversation there. I will. I've been
44:21challenged to clean up my repo and put it in the show and tell channel and we would love to see what you build as well. So please join us in the Slack community. And we'll make sure that we take all of these as follow ups too. Yeah, appreciate the discussion as well, folks. Thanks for that. Yeah. Yeah. Thank you, everybody. Really great question.
44:42Yeah. Steve, thank you for flagging. If that's broken, that is something I can fix with my Claude's very quickly. So thanks. Cool. Anything else, Stu, that we didn't cover that you want to make sure folks know? No, no, this was great. Thank you, everybody, for joining. And, you know, I'll just say, you know, what you saw today was just one prompt to get from, you know, zero to a scheduled pipeline in
45:06life and like in a live dashboard. You know, that's like the first birthday we're trying to hit, right? If you want to try yourself, get on it now. You know, there's it's available to business and enterprise plans or anyone who's trialing the product. And we would love to hear what you build. So, yeah. Amazing. Thank you for joining us. I appreciate your guessing and your demoing. Thank you,
45:30everybody, for coming in the community. And in the meantime, take care.
