Empowering Data Scientists with Zero Platform Friction: Deploying Streamlit & Friends in 3 Minutes

,

In this presentation, Bernhard Schäfer, Senior Data Scientist at Merck, and Nicolas Renkamp, Global Head of Platform Product Portfolio at Merck, address the common "wall" data scientists hit when transitioning a Streamlit or Dash prototype into a production-ready application. Bernhard and Nicolas demonstrate how to eliminate platform friction by showcasing Merck's self-service app service, which enables teams to deploy governed applications in just three minutes. By examining a platform that powers over 750 active apps for 8,000 users, the speakers explain how to integrate essential enterprise requirements—such as SSO, CI/CD, and scoped data access control—without slowing down the development cycle.

The session features a live demo of the end-to-end workflow, from using a web console for provisioning to the automatic generation of deployable Git repositories. Viewers will learn about a cost-effective AWS-based reference architecture and the specific design decisions required for safe multi-tenancy, identity propagation, and "Day-2" operations like rate limiting and idle shutdowns. Whether you are a data scientist looking for a blueprint to request from your IT department or a platform engineer building self-service tooling for a governed environment, this talk provides practical patterns for balancing central control with decentralized freedom. Attendees will walk away with a clear framework for shipping data and AI products at scale while maintaining strict security and reliability standards.

This description was generated by Open-Source AI using the transcript of the session and the original submission contents.

This session took place in track MLOps & DevOps and was classified suitable for intermediate domain / intermediate python by the speaker.

Submission

The proposal as submitted by the speaker before the conference.

This session is for anyone who has built a Streamlit (or Dash, R Shiny, FastAPI, React) prototype and then hit the wall when it needed to be shared with real users: access to live data, SSO, permissioning, deployment, and operational guardrails.

We will present the workflow and the architecture from both sides: as a data scientist shipping an app, and as a platform admin operating the service safely at scale.

What we will demo

We will demo the end-to-end workflow from zero to a running app using our internal app service. The platform includes a web console for self-service provisioning and configuration and the deployment runtime managing the state of the application.

  • Using the web console to create and configure a new app from a framework template (Streamlit, Dash, R Shiny, FastAPI, React).
  • How a Git repository is created and the first version is deployed behind the scenes, including a working starter app with example pages.

Key design decisions (the parts that are usually hard)

  • Identity propagation: the app receives the signed-in user identity from SSO and uses it for downstream authorization.
  • Authorization at the data layer: dataset permissions are scoped to use-case resource, making sure tokens can not be exploited.
  • Safe multi-tenancy: per-app isolation plus resource limits to prevent noisy-neighbor problems.
  • Repeatable delivery: templates plus CI/CD conventions so a new app starts from a working, deployable baseline.
  • Day-2 operations: guardrails like quotas, rate limiting, and idle shutdown to keep the platform reliable and cheap.

Running at scale

  • Production usage: 750+ active apps and 8k+ unique end users (2025).
  • Infrastructure run rate under 10k USD per month (excluding engineering time).

Who should attend

  • Data scientists and analysts who want to ship apps beyond a demo.
  • Data platform and DevOps engineers building self-service tooling for governed environments.
  • Teams standardizing how internal data & AI products are delivered to business users.

Takeaways

  • For data scientists: what a good internal app hosting platform should provide, and which requirements you should ask your platform team for (governed on-behalf of data access, templates, CI/CD, guardrails).
  • For platform teams: a blueprint you can adapt beyond AWS, including the architecture and tradeoffs necessary to operate fine-grained authorization and a multi-tenant runtime at scale.

If you do not have such an app platform in your company yet, use this talk as a checklist to start the conversation with your IT or platform teams. :-)

Transcript (auto)

Auto-generated from the recording utilizing Open-Source AI. Speaker labels (Speaker 1, Speaker 2) reflect diarization, not identity. Timestamps refer to the recording.

Bernhard [00:06]

All right, hi everyone. As mentioned, my name is Bernhard, this is Nikolas. And today we'll show you how we turn deploying a data app at Merck from like a weeks long odyssey to a three minute task. And first of all, who are we? So as mentioned, I'm a data scientist and Nikolas is the global head of platform products portfolio. So he's responsible for Merck's data platform. We're both big fans of the PyData community and we've spoken here at PyData and local meet us before. So it's great to be back. And also some quick facts about Merck. So Merck is a science and technology company and we're the proud Keystone sponsor of this conference. We have more than 62,000 employees worldwide and operating in more than 65 countries. And we actually have also a very long history, more than 350 years and we were founded right here in Darmstadt, so just around the corner. And with that, let me go to the motivation of this talk. So before I joined Merck, I used to work as a data science consultant. And the key challenge I saw in every company was going from a prototype to production. So imagine you developed a Streamlit app, that happened to me quite often, on your notebook, business is excited, but then you hit reality. And so you have to deal with questions such as, how do I deploy this so that end users can access it? How do I get access to live data instead of working with CSV exports? And how do I set up all of these difficult things such as single sign-on and authentication permissions and so on? And so for me, oftentimes, building the prototype, for example, Zimbal dashboard, you could do this in a matter of days. But then getting it deployed oftentimes took weeks, which left me very frustrated. And of course, it also blocks data scientists from shipping apps to end users and iterating with them. And so when I then joined Merck three and a half years ago and discovered Merck's app service that Nicolas and his team built, I was super excited. And what you can see on the left is a screenshot of the app service console where I can manage all of my apps. And here I can just click on launch, then select the template. And then three minutes later, I have a running app with single sign-on, CI, CD, et cetera, and with governed data access. And I will do a live demo in a minute. And what you also see here on the right-hand side are some basic stats. So right now, the app service has close to 700 active apps, reaching almost 3,000 monthly active users. And it costs per active app per month around 21 euro. And Nicolas will later go into all of the architecture details that make it possible to run the platform at such a scale. All right. So let's put it to the test. Please help me cross your fingers. The live demo will actually work. And yeah, so for this, I actually have the app service console here. And this is very hard to actually maneuver. So I can, as mentioned, click on launch and then create a new app. So I have to type in a name, pydata, hello world. And then I can also select a pre-existing template. In this case, I'm actually happy with the Streamlit template. I can configure additional owners, contributors, and so on and so forth. And also very importantly, I have to link each app to a use case. Nicolas will explain more what this actually means. And what I also really like is that it's just a matter of, where is this? Here. It's just a matter of a couple of clicks to, for example, also connect to external APIs. So if I want to use OpenAI, I just select it here. Let me actually select the productive version. Or if I want to have access to Snowflake and need an API key, it's also just one click. And then once I'm done, I'm clicking on launch. And you can now set your timer. So the platform will now take care of the rest in terms of it will create a repository, build the default Docker image, deploy it, and so on and so forth. And yeah, I don't want to now sit there for two minutes and wait. That's why in the meantime, I'm going to walk you a bit through, for example, templates that we can provide. So here you see the latest version of the Streamlit template that we can use for rapid prototyping. And yeah, I'm an active contributor to this template. So I try to make sure it follows, like uses all of the modern PyData tools, such as UV for dependency management, and you name it. And it also comes with a pre-configured Cloud MDs. You can directly start coding. And also, what is very flexible is here you already have a couple of predefined GitHub workflows that, for example, deploy the App Service, run the test suite, and so on. And what I wanted to show you now is a bit more, and this is something that excites me a lot in terms of how easy it is to get started. So here I already created a new repository based off the template and also a new app, so this App Service Usage Data Dashboard. And I linked this to a use case that actually contains App Service telemetry data. And what I want to try now is, while the other app is being deployed, to show you what working with it looks like in practice. So what I can do is I can just, I mean, I cloned it already locally, but didn't do anything else. And now I have already started creating a prompt that says, create a Streamlit page that reproduces the metrics using the App Service KPI monthly data set. Well, the question is, of course, which metrics? And now this will be maybe a bit too ambitious, but what I'm trying to do now is, I will create a screenshot of the stats that you saw before, and then hopefully find a screenshot. Yes, it's here. Copy it, and then also try to paste it into the prompt. And now I can, it will start running. And so what we will do now is, what you see is, it tries to query the underlying data set, get the schema, and also run the sample query to just fetch some rows to get an idea of the structure of the data set. And then, fingers crossed, it should develop the first version of this Streamlit page. Oops. And in the meantime, I can already start my local Streamlit development server. It's here. And then, let's see. Okay, let's allow this. So I think, yeah, I tried now to create a new page. So if I refresh this, and now find my mouse, wherever it is, and access this page, you get, hopefully, yeah, you get the numbers that you saw before on the slide. Go ahead. So what happens under the hood, right? So it fetched the data set from the data platform, and then computed those metrics, and also showed how they compared to last year. And so, yeah, this was basically to just show you what it looks like working with it. But, of course, it has a dual purpose, because I also wanted to, you know, spend the three minutes until the app is up and running. So let's refresh the page. Yeah, it should be there now. So this is the Hello World app that we created. And as mentioned before, it comes with single sign-on already. And we support multiple data platforms as back-end. Here we're using Foundry as the default. And here's the single sign-on page where I, it's just not the first time that I accessed the app, I have to allow it that it can query data on my behalf. And then once I have done that, it will redirect me to the actual page, which also comes with, you know, a lot of sample pages to showcase platform capabilities, such as, yeah, as mentioned, it will already have an open API key for me, so I can directly start using LLMs and so on and so forth. Okay, so much for the live demo. So maybe let's do a quick recap, because I went quite a bit back and forth. So first thing that I think is great is that, as a data scientist, it's great to have a self-service platform where you can just, like, create an app in minutes. And that it already comes with features such as single sign-on and CI CD. It's also, as you saw, just a matter of one click to configure access to external systems and to give it access to LLMs. And I think what's also important is that I'm not being locked in here to just, like, have a template that I can't change. Templates are actually very customizable for your own stack. And as you saw, it also then comes with a generating engineering support. And with that, I'm actually now going to hand over to Nicolas, the magician who developed the first version of this and is responsible for the app service. Thanks, Bernhard. Impressive demo.

Nico [10:04]

I think the demo gods are on our side. One-shotted the same KPIs that we saw on the slides. I'm definitely not a magician, but maybe some of you have heard of this term here or this phrase that any sufficiently advanced technology is indistinguishable from magic. In our case, I have my own version. It's a heavily parameterized CloudFormation template that you saw in action. Of course, that's not the only secret behind the app service, but the core is basically an infrastructure as code template that is heavily parameterized and then also a lot of building blocks that are beneath this template. So in the next 20 minutes, I want to walk you through the architecture of the app service, how we do govern data access at Merck, and why it enables the zero tickets creation process of apps, and as well some key design principles which we follow while developing the service the last four years. So let's start with the workflow when you create an app. You have seen Bernhard accessing the management console, which is a React frontend deployed on S3 and CloudFront, and a serverless Lambda function with NestJS as a backend. So all TypeScript very handy nowadays, but the first version was created by my working student back in 2022, all written by hand. So the management console takes the inputs and deploys a CloudFormation template that will create a CloudFormation stack in AWS. Within this stack, the main piece is of course the container of the application. And in the demo you saw that we created the StreamNet sample app. Afterwards, developers or data scientists can then modify a fully flexible disk container and can deploy any kind of Docker container. Then we have a very lean authentication sidecar next to each app as a container based on Nginx and VouchProxy. And this is a very important piece of the architecture which I will explain more in detail in the following slides. We also have custom resources that are managed to provide integrations into all the other pieces of our ecosystem. One piece that was missing in the previous slide was obviously the data platform because those apps are extensions of data and use cases on our platform. And so the data needs to come into the app or the app needs to be able to write back into the data platform. And when an end user opens an app, we have a shared load balancer that redirects to the ECS service and task of Fargate. Fargate is a serverless container runner, so you don't have to manage any infrastructure. You can just tell Fargate, this is my containers. Give me this amount of CPU and memory and it will run it for you. So we route this request to the authentication sidecar, which redirects to the data platform, where if you remember you saw the dialog where Bernhard needed to press allow. And this is a very important piece of the architecture because it creates a scoped down user token, which is then forwarded by the authentication sidecar into the container. The container can then use this user token with a connector library to fetch data from the data platform on behalf of the user with the correct permissions. You also saw that we talked about use cases, we talked about data, and we talked about apps. And that might sound a little bit abstract, so I'm going to take a step back to explain a little bit how we do govern data access and why it's the fundamental for the scaling of the app service. So I'm sure you have a similar version for your company, how the data generally flows from source to the use cases, to the targets, to the value creation. And in our company, we settled kind of on this model. It's a simplified version, of course. We have the data sources on the left side, which are then ingested into the data platforms. So we have the data sources on the right side, which are then used to the data sources. So we have the data sources on the right side, which are then used to the data sources. And then there is a break and use cases that have a clear intent work based on those data products to produce value for the company. So we have the implicit contract that is being made between the data owners and the use case owners, that the use case owners only use that data for the specified purpose in the use case, and not for others. And that they also take care about the data and what they expose to their use case users. So in this case, we have two example use cases, a sales forecasting app and a quality dashboard. So how does that now link to the app service? Each application in the app service is linked to a use case. And when a user opens the app, even though the user could have theoretically access to both the sales and the quality data set that you see here, when accessing the platform directly, accessing the app, the user token is scoped down to only the resources that are linked to the use case. So the access through the app to the quality data set will be forbidden. This has two consequences. First of all, it enforces our governance model so that the intent of the use case cannot be extended unlimitedly. Second of all, it is a fundamental security. It's the resolution for a fundamental security issue that would disallow scaling of apps or accessing of production data from apps without any review. Which is here the case. We accessed the production data. There was no one in between that looked at the code or whatever. Let me do an excursus why this is important. Imagine a hypothetical situation where an M&A director who is responsible for making acquisitions for a company has their own use case where they list the acquisition targets in a data set. She also has access to the sales forecasting use case because she needs to know how the long term sales of different areas of the company will evolve to understand what is a good target and what is a bad target. In the case, if there is no token scoping, a malicious developer could deploy an app that gets the user token, enumerates all data sets, and exfiltrates the acquisition target data set, and do something that we don't want with this data. In our case, apps only get scoped user tokens. So, they would, because the app is linked to a use case, the tokens are scoped to the projects of this use case, and requests to other data sets would be forbidden and fail. And that is the core principle why we are able to grant, basically, production data access from second zero. Because the use case owner and the app developer, they are basically the same trust principle, and own this data anyway, so they can decide who gets access to it. I also have two slides in the slide deck, how we do that under the hood for Foundry and Snowflake. And then the slides if you want to know more, but out of time reasons we are skipping those slides here. So, the app service started in 2022 as a side project during COVID times. And out of necessity, we had to follow a few patterns to cope with all of the features and demands that people had, and also the maintenance. So, we basically always didn't have time to do any maintenance. So, everything needs to be automated. And I want to share a few patterns from this journey, I would say. So, I think the first one I want to share is that we heavily rely on the infrastructure as code provider, in this case CloudFormation. But it could also be Terraform or other equivalent solutions to do the orchestration of the infrastructure and the dependencies. So, we fully rely on it for the state management. We rely on it for the rollback behavior as something goes wrong, creating apps, deleting apps, and dependency ordering. And we, of course, get audit rates out of the box. We get events out of the box. So, AWS emits events when CloudFormation things change, and we can hook in integrations. A key piece is that we can extend CloudFormation with custom resources. And also here, other infrastructure as code solutions have their own version of custom resources. So, this can also be done with other technology. But in our case, we, of course, have the data platform integration as a custom resource. Azure DevOps repository and pipeline. Cleanup script for the ECR images. Integration into our Bayesian backend API that was also presented last year here at PyCon. As well as our AI and ML services to get access to LLM APIs. And those custom resources are quite easy to implement. They're just Lambda functions that need to react to create, read, update, and delete events. And then those Lambda functions can manage the infrastructure and return back the relevant information to the stack template. Another principle is that we always put the developer experience first. And we go to a very much extent to do that. So, of course, we have the sample apps that you saw. We try to put the developer on the right path by always creating repository and CI-CD integrations. So, it's the official blueprint to deploy apps. You can manually push Docker images as well, but I don't think anybody is really using that. We also integrate with GitHub through OpenID Connect. As I said, we have the ecosystem integrations. We also have our newly added HPC cluster, AWS accounts, you name it. We developed custom developer libraries that abstract away how you interact with our data platforms. And so, you can run the same line of code within the app service and locally. You saw that when Bernard ran the app locally with uvrun. You actually didn't see the code, but there would be no code change required to deploy this app, just a git push to the app. And then it would run completely like that on the app service using the scoped user token from the logged in user. We recently added a conditional firewall to make fine-grained egress possible. And we also have more, let's say, production apps, not the streamlet, the five-minute streamlet app, more production apps. And those also can use DevQA and prod environment and link it to one repository and pipeline. From a platform engineering perspective, we implement quotas from day one. So, everybody can just create a certain amount of apps and memory and CPU, just to avoid getting a denial of service from our own users. Of course, quota increases are possible. We try to emulate serverless behavior in the sense of a request response style serverless, which is not possible today in Fargate. So, what we do is, usually the Fargate task is stopped. And when the first user accesses the app, the CloudFormation template starts the app. The user waits on the starting page. And after a minute, the page refreshes and the app is there. This is very neat and totally tolerable for research apps, apps with a small amount of users, and so on. We also stop apps automatically after 30 minutes if there are no requests. Of course, this can be changed and configured to run all the time. We also have schedules for apps that can start and stop for business hours. But these things keep the costs low and also make no headaches for both me as a platform engineer as well as the data scientist that can just know my app will anyway stop. It's fine. I can leave it running. It will stop in 30 minutes. We share anonymized telemetry data with the app owners so they know how many people are accessing the apps to improve the experience on the apps. We also scan all security, sorry, we also scan all Docker images with Inspector to have security built in. So let me summarize from the platform perspective the key success factors of the app service. The first one is that we put the developer experience first and we make it very easy to do the right thing. Internally, we delegate as much as we can to others to do hard things so we can keep doing simple things and don't have a lot of code in the code base. We have a lot of automations to keep the platform cheap and reliable and we always think about keeping the cost low. But I think the key enabling factor for the scale of the app service is that we early on harmonized the governance model and how data is governed at the company. And because of the use case based governance, we can do the scaling, sorry, we can do the scoping to the use case resources. And thus, we don't need independent code review steps that make sure that the scenario I painted is not happening in the apps. Having said that, I want to hand back to Bernhard for the summary. Yeah, thank you, Nikolas, for the deep dive.

Bernhard [25:29]

Yeah, so let me do the wrap up. So Nikolas just shared his perspective from more like a data platform engineer or owner. But I also want to provide like a conclusion for more like a data scientist. So for me, if I were to, you know, join another company, I would have kind of a mental checklist of what I would like to have to, for me, in terms of developer experience, so that I can quickly develop streamlit apps and not get stuck as I motivated in the beginning, as I often saw before. So ideally, you want to have a self-service app platform where you can create your own apps in a matter of minutes. And they should already come preconfigured with single sign-on and things like CI CD, because I think oftentimes as a data scientist, this is like, you know, not your core expertise. So it's great if you already have this built in and also have then, yeah, templates that you can use. Nikolas also explained quite in depth the concept of scoped user tokens. And I also think this is great that I don't have to think about, yeah, how do I develop the app such that it, you know, only accesses the data that it's supposed to. But I can really rely on the platform here and this scoping so that it will only access the resources that it's supposed to. And then I think as a data scientist, I want to, you know, use modern tools and platforms. So it's great to have a template that can give me already a working app on day one. But I think for me, it's also important that I can customize these templates and make and also, yeah, customize them to the full extent. So you saw the GitHub repository before. I can change anything that I want there in the GitHub workflows or change what I use for dependency management and so on. So I don't have this login and I can always stay on, you know, kind of the cutting edge and even contribute back to the overall templates. Yeah, so these are the four main points. And before we wrap, like before we, yeah, come to the end, I also wanted to last but not least mention one more thing that I'm really excited about. So in my opinion with tools like Cloud Code, way more people can now build apps. And you saw this in the demo. With the right template, it's one prompt to produce a working streamlit page. And at Merck, we have a fast track upskilling program that trains domain experts, so people with very limited coding experience, to become app builders. By the way, shout out to Boris. Where is he? Yeah, he's actually over there who is responsible for this program. And as a mentor, I experienced firsthand here how fast you can, people can actually develop their first app with a good template and tools like Cloud Code. And so I'm convinced that this will accelerate the growth of the app service even further. So I think this growth trajectory will continue. But of course, this only works if you have a scalable platform with a great developer experience and these strong security guardrails behind that. But yeah, and I think with that, yeah, I want to conclude our talk. And thank you for listening. By the way, we also have a talk in the afternoon today by Nils and Andreas, two colleagues from MAC who will present Octopus AutoML. They developed the framework for extracting signal from small and high dimensional data. And also, of course, reach out to us anytime in the conference if you have questions that won't be answered in the Q&A.

Chair [29:38]

Thank you so much. That was an insightful talk. And the next one also sounds interesting. Well, let's move on to the Q&A session. We have quite some time. Let me refresh. Maybe some new questions come up. OK. First question. How do you manage security updates of your containers? So we have security boundaries around the containers.

Nico [30:10]

So they cannot really talk to anything except the allow listed host names. That is kind of a parameter for us. We are rolling out GitHub Actions Depender Bot integration. So we want to start early there, updating the dependencies. So we are enforcing, we want to start enforce log files with the recent supply chain attacks. But I would say we don't have a fully developed and implemented solution across the company. Right now it's the app owner's responsibility. But we are trying from the platform side to find a good compromise between flexibility and speed and security. And yeah, the boundary of the application when it's running is quite strong. So even though there would be an infection, we do think that the exfiltration would not work. But at the same time, we also want to, as I said, start at the source with Depender Bot and other tools that the CI and version control providers provide for us. And also the apps by default don't have internet access.

Bernhard [31:35]

So, you know, these recent supply chain attacks that just then upload data somewhere to the internet won't work. And yeah, that's just one point to add.

Chair [31:45]

Is it something that you can think of for like future releases, a feature that you can have in the future? Yeah.

Bernhard [31:52]

I mean, we also saw the talk on the first day. I mean, where, I forgot who was it, but somebody actually introduced a framework that tracks exactly this issue in terms of how do apps and repositories, which ones already have the latest security fixes and so on. And so I think this would be also a great addition to the platform.

Chair [32:14]

Community always helps. Next question. Where do you draw the line between letting people spin up a Streamlit app for KPIs versus serving them via BI solution like Power BI?

Nico [32:33]

So we like to give people options and we don't, and then it depends basically on the strategy of the sector data office in which direction people are going. But centrally, we try to create the best offering, like a menu, and you can pick from the menu. But we don't force anyone into any direction. So they are, personally, I think it really comes from the skills and the appetite. If someone is trained and has the appetite to build their own complete apps, then we don't want to stop them. If they rather want to build Tableau dashboards, fine for me as well. I think, also personal opinion, I think more and more will go to these kind of purpose-built apps. What I have seen also how good it works, I think there will be more templates, there will be more structure that BI apps can be implemented as individual apps and less into tools like Power BI or Tableau. Thank you.

Chair [33:39]

Next question. Why do you use CloudFormation instead of other infrastructure as code tools like Travform or OpenTofu?

Nico [33:50]

So there's no real technical reason why we use CloudFormation. It's more that in the company, we already had a few predecessors of those services and that use CloudFormation. Once you start with something, it's already, as I said, launched in 2022 or four years old. So yeah, it's more a decision and not based on any kind of underlying technology factors. Yeah.

Chair [34:24]

Next question. Where do you, I guess we ask this question. All right. This one. How do you do lifecycle management for the 700 plus apps? Is this fully automated? And how do you avoid breaking things like doing updates? Yeah, that's a very good question.

Nico [34:52]

So we always, whenever people update their app, we update them to the latest template. So the template has to be forward compatible. So whenever we do changes, we do them in a forward compatible style. So we can just do additions basically. So when you don't update your app, you're running with this version that is around. Every app, when it starts, it gets the latest Nginx and Vouch proxy. So the sidecar will always update. Also there, we only do forward compatible changes. But these updates are always isolated to the app. So we rarely do any changes to the surroundings that are in the hot path of the request of the user. So we don't really change anything in the ALB or in other pieces. So when something breaks, it's probably in an individual app. And we have a DevQA and prod environment for the app service. And we would notice earlier. When it comes to the 700 apps, to be honest, we sometimes do cleanups, campaigns. But we just leave it to the users. Because a stopped app is, well, I don't care. A stopped app is a stopped app. There are almost zero cost for a stopped app. And so they can live. Sometimes every year or so, we send out some emails. So I can say you didn't open your app in a year. You didn't update it. Maybe you can delete it. But I don't really care.

Bernhard [36:30]

Yeah, and maybe to also add to that, since we also have these quotas, for example, by default, you can only create 10 apps. That also encourages me as a data scientist to also clean up once I reach that limit, of course. I can also delete apps in the app service console. And also, you have, they are connected to these use cases. And so, use cases also have a lifecycle. And it is also attached to that. Yeah, the use cases have a lifecycle.

Nico [36:58]

They have owners. They have compliance checks. All of that is handling on the use case level. Thank you.

Chair [37:06]

Next question. Do you have a kind of data catalog so use case owners can see what's available?

Nico [37:21]

Yes, we do have a data catalog. And there, the data is cataloged. And then, when you request access, you need to select your request access for a use case. And that's how people, use cases, get access to data. Okay, yeah.

Chair [37:38]

Next question. You use Streamlit and you use Snowflake. How you also investigated the Streamlit integration in Snowflake for this? Yes.

Nico [37:52]

So, the thing launched in four years ago. Snowflake didn't even buy Streamlit. And there is just a lot of things we built that would be very hard to rebuild on top of Snowflake. And then, it would be only possible for Snowflake as a backend. And we have multiple other platforms. So, we investigated it. And my personal opinion is that from running Streamlit in there, it's now, I would say, production ready. They started with a quite subpar experience where some features of the open source Streamlit did not even work in Snowflake and Streamlit. Yeah. I would say if you, that's also something we wanted to say, right? If you have a platform, it's probably maybe Snowflake, Databricks. They are all offering features that you could use to build something similar. Snowflake has Databricks, apologies, Databricks has Databricks apps, which also allow token scoping. Not to that extent, but I've heard they are working on it. Snowflake, you can attach a role and then you can scope this role down to the individual resources. Thank you.

Chair [39:05]

Next question. Some of your data platforms, to my knowledge, at least Palantir Foundry, include ways to deploy custom web apps within UIs from within the platform. What are the added benefits of your platform compared to that? Mm-hmm.

Nico [39:24]

The similar question to the one before. This one is about Foundry. The one before was about Snowflake. Again, four years ago, those capabilities were not existing. In Foundry, you can deploy web apps, so React apps or anything that can be a web app natively, but you cannot deploy container backends. So it's also not the same. Yeah.

Bernhard [39:50]

Yeah, I mean, from a data scientist perspective, I can also say that with this platform and given that I can use flexible, I can use GitHub, which is like best in class for managing repositories. I can use GitHub Actions and I feel like everything that, so I think the whole development workflow uses these best in class systems, whereas typically if you use the more like existing offerings from vendors, then in my experience, you typically don't have such a great development experience.

Nico [40:23]

Yeah, that's a super good argument that I also wanted to say that if you only pick the one from your platform provider, you might have to make compromises on picking best of free technologies. So you might need to use your repository in the proprietary platform, or you might need to use, maybe you cannot use a Docker container, but you have a different, yeah. So there are many arguments to having something on the side.

Chair [40:50]

I think we also have a similar question if we get the time to reach to it. Next question. What does the change process for apps deployed on the platform look like? Think Dora.

Nico [41:02]

Not sure what Dora stands for. Me neither. But the change process is we create a default pipeline, CICD pipeline, and then Bannert can fully customize it. So they can invent their own change process, have approval gates or whatever is needed, or can be implemented.

Chair [41:31]

If you want to explain your question, I can pass on the microphone. All right. Next question. Okay. Great talk. How many people maintain the platform and build new templates? Yeah.

Nico [41:47]

So for maintenance, we have one full-time developer recently added. Before that, yeah, maintaining is very low effort. So it was kind of part of a responsibility of one of my teams on the site. But now we have one, due to the growth, we have one dedicated developer. And we are working next on kind of a real app store where it's easy to register more templates. And where those templates can then be more easily in a source than today, than knowing me or others, to make it really scale across the company. So, yeah. All right. Something similar I see. Okay.

Chair [42:36]

I just pass it. Did you evaluate or can you compare with MS Power Platform, Microsoft Power Platform?

Nico [42:49]

I can't. No. I'm trying to stay away from that. That's a good answer. Yeah.

Chair [42:55]

We all agree on that. Next question. Is this service open source? No.

Nico [43:04]

Sorry. Many of the things would be Merck specific. But I think we gave a great detail here, how you could build such a service. Also, we have the two slides for the scoping. Yeah. So, but yeah. That's the way it is. I guess we have time for...

Chair [43:30]

But our baby package is open source.

Nico [43:32]

So if you are in the Bayesian optimization space, yeah. Nice. Baby. Baby.

Chair [43:40]

Oh, they actually asked to see the code on GitHub. Yeah.

Bernhard [43:48]

So you can find anything. Yeah. So yeah, if you want, I mean, I think we were, we tried to be very open about all of the details here. Of course, you can also approach us after the talk if you have more detailed questions. Yeah.

Nico [44:01]

We are at the booth. It's, yeah, not to miss. Yeah. And we also have some merch, if you want.

Chair [44:07]

I just read out another question. Final question. Let me go for this one. What is the maintenance budget for the app service or templates? How much effort is it to add new use cases or components?

Nico [44:31]

There is no, so to say, maintenance budget. As I said, it's, if everything, if nothing changes, it's more or less zero maintenance. And the templates, we try to work together with our data and I community and have inner source approach. So there's no dedicated, everyone can, if they want, contribute. Otherwise, the templates are as they are. Of course, if there are security issues, we will fix it. But yeah, there is kind of a dynamic to get those templates to the next level. Thanks to also Bernhard, for example.

Chair [45:15]

Thank you very much for the insightful presentation. Let's thanks again our speaker.

Bernhard Schäfer

Bernhard is a Senior Data Scientist at Merck with a PhD in deep learning and over 7 years of experience in applying data science and data engineering within different industries. For more information you can connect with him on LinkedIn. 🙂

Nicolas Renkamp

As the Global Head of Platform Products Portfolio, Nicolas leads high performing teams that design, implement and maintain Merck's global data, analytics and AI ecosystem UPTIMIZE.

Social card for talk: Empowering Data Scientists with Zero Platform Friction: Deploying Streamlit & Friends in 3 Minutes