Why I Love Modal

2.1 The Pain of Traditional Deployment

The Runpod developer experience was genuinely painful. You had to:

1. Write a Dockerfile locally
2. Build it on your machine (or rent a GPU instance just for building!)
3. Push massive images to a registry
4. Configure everything through their web dashboard

The worst part? Model weights were typically bundled into Docker images, creating 50GB+ monsters that took forever to build, push, and pull. Want to tweak a hyperparameter? Rebuild the entire image. Need to update an environment variable? Back to the dashboard.

Replicate was simpler — no Dockerfile required — but came with rigid constraints. Your code had to fit their exact structure:

			#Replicate'srigidstructure classPredictor: defsetup(self): #Loadmodelhere pass  defpredict(self,prompt:str)->str: #Yourlogichere,butitmustfitthispattern pass
		

This worked for simple cases, but complex workflows? Forget about it.

2.2 Then Came Modal

When I discovered Modal, the difference was immediately obvious. Here's is an example of a Stable Diffusion deployment, using various Modal features we will cover in this post:

			importmodal  #DefinetheenvironmentinpurePython image=( modal.Image.debian_slim(python_version="3.11") .pip_install("torch","diffusers","transformers") .pip_install("xformers",gpu="A10G")#GPU-optimizedbuild )  #Getavolumebyname,toavoidredownloadingthemodel #Createitifitdoesn'texist model_volume=modal.Volume.from_name( "sd-models", create_if_missing=True )  #Getasecretbynamefrommodal huggingface_token=modal.Secret.from_name("huggingface-token")  app=modal.App("stable-diffusion",image=image)  @app.cls( gpu=["A10G","A100:40GB"],#RunonA10GorA100(improvedisponibility) volumes={"/models":model_volume},#Mountavolumeformodelcaching secrets=[huggingface_token],#Injectsecretsintothecontainer container_idle_timeout=300,#Keepwarmfor5minutes enable_memory_snapshot=True#Enablememorysnapshots ) classStableDiffusion: #Thisrunsonceandgetssnapshotted #Thiscansaveupto10soncoldstarts @modal.enter(snap=True) defload_model(self): importos fromdiffusersimportStableDiffusionPipeline  #LoadthemodelfromtheHuggingFacemodelhub #Thiswilldownloadthemodeltothevolumethefirsttime #Subsequentrunswillusethecachedvolume self.pipe=StableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", cache_dir="/models", token=os.environ["HF_TOKEN"] )  #Thisrunsfromsnapshot,movesmodeltoGPU @modal.enter(snap=False) defmove_to_gpu(self): self.pipe=self.pipe.to("cuda")  @modal.method() defgenerate(self,prompt:str,steps:int=20): image=self.pipe(prompt,num_inference_steps=steps).images[0] returnimage  #Youcanalsodefinemultiplemethodsthatwillusethesamemachinetype(saveoncoldstart) @modal.method() defgenerate_with_lora(self,prompt:str,steps:int=20,lora_path:str): self.pipe.load_lora_weights(lora_path) image=self.pipe(prompt,num_inference_steps=steps).images[0] self.pipe.unload_lora_weights() returnimage  @app.function() @modal.web_endpoint(method="POST",docs=True) defapi_generate(prompt:str,steps:int=20,enable_lora:bool=False): sd=StableDiffusion() ifenable_lora: image=sd.generate_with_lora.remote( prompt, steps, lora_path="path/to/lora" ) else: image=sd.generate.remote(prompt,steps) ... return{"status":"generated","prompt":prompt,"image":image}  #Bonus:Scheduledaweeklyreport @app.function(schedule=modal.Period(days=7)) defgenerate_weekly_report(): ...  #Localentrypoint,exposeafunctionrunnablefromtheCLIwith #`modalrunstable_diffusion.py::run_batch_job` @app.local_entrypoint() defrun_batch_job(): my_list_of_prompts=[...]  #RuntheAPIinparallelforeachprompt forresultinapi_generate.map(my_list_of_prompts): print(result) 
		

That's it. No Dockerfile. No registry. No dashboard configuration. Just Python code that runs in the cloud.

The experience was refreshingly simple:

1. ✅ No Dockerfile needed — just Python dependencies
2. ✅ No manual GPU setup — Modal handles the hardware
3. ✅ No complex orchestration — scaling and monitoring built-in
4. ✅ No registry pushes — changes deploy instantly
5. ✅ No rigid structure — full flexibility in my workflow

2.3 Comparison with Traditional Serverless Platforms

What really stood out was how Modal preserved my existing workflow. I didn't have to restructure my code or learn a new paradigm — I just added a few decorators and my local code became cloud-ready. I could even test locally and deploy the exact same code to the cloud.

Just Python. I wrapped my existing Stable Diffusion code into Modal functions and deployed it — within minutes, I had a running GPU endpoint that was faster and more reliable than anything I'd previously deployed.

Tip

Modal makes GPU APIs as easy to deploy as a FastAPI route — exactly the kind of fast feedback loop that data teams need.

2.4 What I Use Modal For Now

Since that first deployment, Modal has become my go-to for:

• Internal APIs — Quick endpoints for team tools and dashboards
• Scheduled ML jobs — Daily model retraining, data processing pipelines
• Prototypes and production endpoints — From proof-of-concept to customer-facing APIs

Each time, it scaled with me. Each time, it just worked. Each time, I experienced what Erik envisioned: infrastructure that gets out of your way so you can focus on the actual work.

Important

The best infrastructure is the kind you don't have to think about. Modal delivers exactly that experience.

4.1 Why It's Great

• Remote Builds Modal builds containers in the cloud — so your laptop can stay cool.

• Layer Caching Only the changed layer is rebuilt. Fast iteration, every time.

• Local File Attachments Add local scripts, configs, or whole packages — all from Python.

• Reproducible Runs Every function runs in a clean, identical container. Goodbye "works on my machine."

You're not locked in either — Modal also supports:

• Custom base images (e.g., from Docker Hub)
• Extending your own Dockerfile
• Hybrid approaches using .pip_install, .run_commands, .env, etc.

Tip

Modal containers are fully declarative: what you see in Python is exactly what you get in production.

Important

You never have to open Docker Desktop again. Modal gives you Docker power, minus the Docker pain.

You can even generate a image procedurally in Python, while a Dockerfile is a static description of the image.

5.1 Why It Works So Well

• Easily Swappable Change the name — not your code.

• Workspace Scoped Share across your team, projects, and functions.

• Safe by Design Secrets are encrypted, scoped, and never persist where they shouldn't.

You can create secrets via:

• Modal dashboard UI (pre-built templates for Mongo, HuggingFace, etc.)

• Modal CLI:

  Copy

  			modalsecretcreatehuggingface-token
  		

• Or dynamically in Python, e.g. from a .env file:

  .env loader

  Copy

  			@app.function(secrets=[modal.Secret.from_dotenv()]) defsecure_fn(): ...
  		

Note

Modal treats secrets like first-class citizens — no plugins, wrappers, or hacks required.

Important

Secrets are injected cleanly, stored securely, and scoped smartly. All you do is write Python.

6.1 Volumes — Ephemeral, Fast, Commit-Consistent

Think of modal.Volume as a distributed scratch disk — a shared folder that multiple Modal functions can read from and write to:

			vol=modal.Volume.from_name("my-volume")  @app.function(volumes={"/models":vol}) defwrite_file(): withPath("/models/weights.bin").open("wb")asf: f.write(...)#Writetothevolume vol.commit()#Committhechangestothevolume
		

6.2 What makes volumes great?

• ⚡ Fast Access — Designed for high-speed reads across workers
• 🧠 Great for ephemeral data — model checkpoints, logs, outputs
• 🔁 Cross-function Sharing — multiple functions can use the same volume

Tip

.commit() is required to persist writes across functions. Think of it like a distributed save button.

6.3 CloudBucketMount — Mount S3, GCS, or R2 Directly

If you want to bring your own storage, you can use modal.CloudBucketMount to mount S3, GCS, or R2 directly.

			@app.function( volumes={"/my-mount":modal.CloudBucketMount( bucket_name="my-s3-bucket", secret=modal.Secret.from_name("s3-creds") )} ) defread_data(): print(Path("/my-mount/file.txt").read_text())
		

8.1 FastAPI Compatibility — First-Class

The @modal.fastapi_endpoint decorator wraps your function in a real FastAPI app behind the scenes, giving you:

• ✅ Type annotations and input validation
• ✅ Auto-generated OpenAPI docs
• ✅ Support for query params, POST bodies, or Pydantic models

			@app.function() @modal.fastapi_endpoint(method="POST") defgreet(name:str): return{"message":f"Hello{name}!"}
		

Need more flexibility? Use:

• @modal.asgi_app() for full FastAPI, Starlette, etc.
• @modal.wsgi_app() for Flask, Django
• @modal.web_server(port=7860) for Streamlit and custom apps

Tip

Modal supports full web frameworks — not just endpoints. Your whole app can live in the cloud.

8.2 Serverless and Scalable

Every endpoint:

• Scales with traffic — from zero to many containers
• Launches in isolated environments
• Optionally runs with GPUs
• Cleans itself up when idle

You don't manage servers or scaling. Modal takes care of all the boring parts — reliably.

8.3 Security Built-In

Want to restrict access? Just add:

			@app.function() @modal.fastapi_endpoint(requires_proxy_auth=True) defadmin_tools(): return"Restrictedaccess"
		

This will add a basic auth layer to your endpoint

			exportTOKEN_ID=wk-... exportTOKEN_SECRET=ws-... curl-H"Modal-Key:$TOKEN_ID"\ -H"Modal-Secret:$TOKEN_SECRET"\ https://my-secure-endpoint.modal.run
		

For advanced needs, you can still use FastAPI's native security (OAuth2, JWT, etc.) — it all works the same way.

Important

Modal's web endpoints turn Python functions into production-ready APIs — with autoscaling, FastAPI docs, and zero maintenance.

9.1 Keep Containers Warm

Avoid spinning up cold containers altogether by keeping a pool ready:

			@app.function(min_containers=2,buffer_containers=2) deffast_api(): ...
		

• min_containers: always keep N containers warm
• buffer_containers: pre-warm extra containers for traffic bursts

You can also delay container shutdown with:

			@app.function(scaledown_window=300) deflong_tail_fn(): ...
		

This keeps the container alive for 5 minutes after the last request — perfect for bursty workloads. This is based on the assumption that if a user just made a request, they will make another one in the near future.

9.2 Memory Snapshots — The Killer Feature

You can go one step further: snapshot the container memory after warmup and reuse it for future cold starts.

			@app.cls(enable_memory_snapshot=True,gpu="A10G") classEmbedder: @modal.enter(snap=True)#HereweimportlibrariesandloadmodelsfromdisktoRAM defload_model(self): self.model=load_model_to_cpu()  @modal.enter(snap=False)#HereweeventuallymovemodelsfromRAMtoVRAM defmove_to_gpu(self): self.model=self.model.to("cuda")
		

This will:

• Run the snap=True hook first, and save the state of the container as a snapshot (ie, all the memory allocations).
• Run the snap=False hook second from the snapshot.

The next time you call the function, it will directly start from the snapshot and skip the snap=True hook.

This is based on CRIU under the hood², the CRIU and Nvidia team are currently also working on the ability to save VRAM state as well. This will be a game changer at this could basically eliminate the cold start time ^{3 4}.

10.1 Workspaces

A workspace is your team's shared space for:

Everyone in the workspace can access shared resources — without having to copy-paste credentials or redo infrastructure.

10.2 Environments

Environments help you separate:

• dev
• staging
• prod

Each with isolated logs, schedules, endpoints, and secrets.

			modaldeploy--namemy-app--environmentstaging
		

Note

Modal environments are optional — but powerful for teams managing multiple pipelines or app states.

11.1 When You Do Want Control

You can explicitly select cloud and region when needed — for:

• Low latency inference
• Data residency & compliance
• Cost optimization (e.g., egress near your storage)

Here's how to do it:

			@app.function(cloud="gcp",region="us-west1") defmy_fn(): ...
		

Modal instantly runs your code on GCP in the us-west1 region — no provisioning needed.

11.2 Supported Clouds

You can choose from:

• "aws"
• "gcp"
• "azure" (coming soon)
• "auto" (default — Modal picks best location)

Important

You get cloud-level control only when you want it. Otherwise, Modal optimizes for performance and availability.

Important

Modal gives you a fully managed experience, but when you need to fine-tune your compute location — you can. The result? Serverless that scales globally, but respects your constraints.

11.3 Built-In Debugging and Monitoring

...

12. Conclusion

Modal has fundamentally changed how I think about deploying and scaling applications. By eliminating the friction between local development and cloud execution, it embodies Erik Bernhardsson's vision of fast feedback loops that make data teams truly productive.

Whether you're building ML inference endpoints, running scheduled data pipelines, or prototyping with GPUs, Modal's Python-first approach means you can focus on your code rather than wrestling with infrastructure.

Note

The Serverless Python Ecosystem: Modal isn't alone in this space. Beam Cloud offers a similar Python-native serverless platform with their own custom runtime, and they've open-sourced the underlying engine as Beta9 for self-hosting. If you're looking to self-host, this might be for you. However they still miss some of the features that Modal has.

If you've been putting off that deployment because the infrastructure feels too complex, give Modal a try. It might just be the missing piece that turns your side project into something you can actually ship.