Learn
Your product 🤖 AI-on 💻 On your computer ⏱ ~16h Difficulty 4/5 v1

Build and ship your own AI app

Build the app you planned in quest 1, from scratch, with your coding agent: logins and roles, your records and one core workflow, and an AI assistant that answers from your data and documents and changes things only after the user confirms (an MCP server is optional). Then put it on the internet (Render and Aiven's free PostgreSQL) with production settings, secrets in the host's environment, rate limits and a daily budget. Finish with honest mini evals, a threat model, a demo and a retrospective.

Join free to start → Read the brief and materials. Join Build and Ship an AI App to start, submit and ask the tutor. Log in

Why this quest

This is the quest the course exists for. You build your own app, the one you planned in quest 1, from scratch, with your coding agent, and you put it on the internet. It has logins and roles, and an AI assistant that answers from your app's data and documents and makes changes only after the user confirms. Adding an MCP server, so people can use it from Claude or Codex too, is optional.

Everything you practised comes together: planning, briefing an agent, checking its work, tool calling, permissions, confirmation, and now also deployment, secrets, cost controls, evals and a threat model.

The four parts

Part What you do In SUBMISSION.md
1. Brief version 2 Update your quest-1 brief: what changed, and why Brief version 2
2. Build Build the app within the envelope, on the course's stack My app
3. Ship it Deploy it on Render with Aiven PostgreSQL, safely and cheaply Live app, Deployment and security
4. Show it Mini evals, a threat model, a demo, a retrospective Mini evals, Threat model, Demo, Retrospective

It is one quest with one submission. Plan about 16 hours over two or three weeks.

The rules

The product envelope (from quest 1). Your app must have:

  • logins with 2 or 3 roles (Django's authentication; we suggest one Django group per role), with a meaningful permission boundary between them;
  • a small data model: 1 to 3 main record types and one core workflow done well (plus one closely related supporting workflow if it needs one);
  • an AI assistant that answers from the app's data (with the logged-in user's permissions) and from a small set of your organisation's documents, with citations, and makes at least one kind of change, only after the user confirms.

Optional: an MCP server at /mcp where each user logs in with a personal token and gets exactly their web-app permissions, as in quest 5. Set mcp: true in app_contract.yaml if you build one; its tests then count. No extra credit for adding it, and nothing is lost without it.

Out of scope: payments, outside services (except the model), sending email or SMS, mobile apps, real-time features. Your mentor may send the app back if it grows past the envelope.

The stack (the same as the whole course):

Layer Required
Backend Django 5.2, Python 3.10 or newer (the grader uses 3.12)
Database DATABASE_URL: SQLite on your computer and in the tests, PostgreSQL in production
Frontend Django templates, plain JavaScript and CSS. No framework, no build step
Model The official openai SDK, Chat Completions, gpt-6-luna, your own tool loop. No agent frameworks, no other model SDKs, no requests or httpx
MCP (optional) A Django view at /mcp (like quest 5)
Tests Django's test runner

The grader installs exactly these packages (the starter's requirements.txt): Django 5.2.17, openai 3.19.2, PyYAML, WhiteNoise and dj-database-url. gunicorn and psycopg are in the file for the server only; your code must not need them in the tests. If your app needs another package, it will not load in the grader. Ask your mentor first.

Your API key never goes in your code. It comes only from the environment variable OPENAI_API_KEY: a gitignored .env on your computer, the Environment tab on Render. The hidden checks fail a submission with a key (or the [REDACTED] marker the platform leaves where it removed one) in the code, the settings, the tests or render.yaml. If a key leaks, delete it in your OpenAI dashboard at once.

Part 1: Brief version 2

You have now built each piece in the course app. Re-read your quest-1 brief and write version 2 in SUBMISSION.md: keep it short (about one page), and focus on the changes.

  • What did you cut, merge or add (roles, record types, workflows, documents)? Why?
  • Your final list: roles, 1 to 3 record types, your core workflow, which assistant tools serve it, and which tool makes a change.
  • Your example interactions: grow them to 5 to 10. They become your mini evals in part 4.
  • Anything your mentor asked you to change in quest 1.

Cutting is a good sign. "I dropped the reports workflow because it is a form, not an assistant task" shows judgment.

Part 2: Build

Build the app from an empty folder with your coding agent. Start with a project instruction file (AGENTS.md, CLAUDE.md or GEMINI.md): the stack, what not to use, the commands, and the rules below. Keep it up to date; it is graded.

Your app must follow the course's conventions, because the hidden tests use them:

What The convention (from quests 4 and 5)
Chat API POST /assistant/api/messages with JSON {"conversation": null or id, "text": "..."} answers {"conversation", "reply", "actions": [{"id", "summary", "status": "pending", ...}], "stopped"}. Login required: 401 JSON for visitors. Another user's conversation: 404. Empty text: 400.
Confirm, cancel POST /assistant/api/actions/<id>/confirm and .../cancel answer {"action": {"status": "done" / "failed" / "cancelled", ...}}. Only the user who asked: others get 404 (or 403). Not pending any more: 409 (or another 4xx). An action runs at most once.
The model OpenAI() created when a message is sent (it reads OPENAI_API_KEY and OPENAI_BASE_URL); client.chat.completions.create(model=settings.ASSISTANT_MODEL, ..., tools=[...], reasoning_effort="none"); ASSISTANT_MODEL defaults to gpt-6-luna.
The loop Every tool call gets a result; problems (unknown tool, bad JSON, wrong arguments, refusals) become a result with the word error, never a crash; a step limit sets "stopped"; a model error answers JSON with "error" (e.g. 502), never 500.
Tools Run as the logged-in user, never as an admin; no tool has a "user" argument. A write tool only creates a pending action; its result tells the model the change waits for the user to confirm. A change that cannot work is refused at once.
Documents A search tool returns sections with their file name, so answers cite it, e.g. [borrowing-rules.en.md]. Keyword search (like BM25) is enough.
MCP (only with mcp: true) As in quest 5: POST /mcp, JSON-RPC 2.0, protocol 2025-06-18. Authorization: Bearer <token>, otherwise 401 with WWW-Authenticate: Bearer; a browser session is not enough; an Origin from another site gets 403. tools/list shows only the tools the user's role may use, with inputSchema and annotations.readOnlyHint: true on read tools; a hidden tool called by name is refused (JSON-RPC error -32602). Write tools create a pending action to confirm in the web app. Tokens are stored hashed.
Commands python manage.py seed_demo creates the demo users and data (safe to run twice). With MCP: python manage.py mcp_token <username> prints a new token as the last word of its output.
Health GET /healthz (no login) answers 200 with the value of NEUGEN_VERIFY_TOKEN in the body.
Limits A rate limit per user on chat messages (429), a daily budget for all users together, counted from the tokens in each response's usage (503 or 429 when it is used up, without calling the model), and a maximum message length (at most 10,000 characters; longer: 400). Keep the counters in the database or Django's cache, not in a Python variable.
Settings DATABASE_URL configures the database. DEBUG, SECRET_KEY and ALLOWED_HOSTS come from the environment.

Your own tests: at least 10, all passing with python manage.py test, using the course's fake model for the assistant (as in quest 4: from neugen_fixtures.mock_llm import MockLLM, reply, tool_calls, with .neugen/shared on the path). No test may need a real key or the network.

Working with an agent on a bigger codebase. This app is larger than anything you built in the course. Keep the agent effective:

  • One layer at a time: models and seed_demo, then the service functions with tests, then the pages, then the assistant, then production settings (then MCP, if you add it). Commit after each.
  • Put the rules in the instruction file, not only in chat: "every operation is a function that takes the user first", "write tools only create a pending action", "no key in code".
  • Ask for a plan before each layer. Read every diff. Agents often add a framework, run a write tool at once "to make it simpler", or use a superuser in a tool.
  • Keep notes of what you checked and decided: you need them for the retrospective. Transcripts in .neugen/logs/ (neugen log <file>) are optional.

Use your app as your users would. Tests prove the rules; using the app shows you whether it helps anyone. Not graded, but stop at each of these:

Try it: after the pages and logins. Log in as your main user and do the core workflow from start to finish with the pages alone. Then log in as the second role and try the thing the main user may not see or do.

Notice: if a step is awkward by hand, it will be awkward for the assistant too. Fix it in the services and pages first.

Try it: after the assistant. Do the same job by talking to the assistant. Ask about your own data and check the answer against the page; ask a document question and open the document; ask for the change, and look at the page before and after Confirm. As the other role, ask for something that role may not do.

Notice: which parts are easier with the assistant, and which are still better as a button? That's your Brief version 2 coming true, or not.

If you can, also watch one other person (ideally the user from your brief) try the core workflow while you stay quiet. Where they hesitate is the next thing to fix.

app_contract.yaml: describe your app for the tests

The hidden tests are the same for every student. Your app_contract.yaml (next to manage.py) tells them how to use your app: your demo users, one read tool, one document question, your write tool, one call a role may not make, and your rate limit. Everything must be true on a fresh database after migrate and seed_demo. The tests log in as your demo users directly, so they need no passwords.

contract: 1
app: "ShedDesk"
mcp: true                       # false (the starter's default) if you skip the optional MCP server

roles:                          # each role: the demo user seed_demo creates for it (2 to 4 roles)
  member: ana
  volunteer: camila
  coordinator: ines

read:                           # a read tool, called as `user` with `arguments`
  user: ana
  tool: list_my_loans
  arguments: {}
  shows: "Hedge trimmer"        # text that MUST be in the result: this user's own data
  hides: "Cordless drill"       # text from ANOTHER user's data: it must never reach the model

document:                       # your document search tool
  user: ana
  tool: search_documents
  arguments: {query: "how long can I keep a tool"}
  file: borrowing-rules.en.md   # the file the answer cites (it must be in your project)
  fact: "A loan lasts 7 days"   # text in that file that the result must contain

write:                          # your write tool: a pending action, the change only after Confirm
  user: ana
  tool: reserve_tool
  arguments: {tool_code: "L-07", start_date: "{date+10}", end_date: "{date+12}"}
  changes:                      # how the tests see the change: the rows this filter selects
    model: library.Reservation  # app_label.ModelName
    where: {tool__code: "L-07", member__username: "ana"}

refused:                        # a call this user's role may NOT make (any tool)
  user: ana
  tool: suspend_membership
  arguments: {username: "luis", reason: "Two tools are late"}

limits:
  messages_per_user: 20         # 5 to 100 for the tests; the next message gets 429
  per: hour                     # minute, hour or day
  • {date+10} becomes the date 10 days from today (YYYY-MM-DD); {date} and {date-3} work too.
  • shows and hides must both be in your seed data. Choose hides from something private.
  • changes.where is a Django filter. Before Confirm, no row of changes.model may change; after Confirm, the rows it selects must be different (a new row, or a changed status).
  • A refused call may add rows to tables that also change for harmless messages (conversations, messages, usage counters), but none of your records may change.

Run neugen test often. It runs the public part of the contract suite on your project.

Part 3: Ship it

Deploy your app on Render (a free web service) with Aiven (a free PostgreSQL database). The starter has a render.yaml and a build.sh to adapt. Checked on 2026-09-29; free plans change, so read the linked pages again when you deploy:

  • Render free web services sleep after 15 minutes without traffic. The next visit wakes them, which takes up to about a minute. The course's live check waits for that.
  • Render's free filesystem is wiped on every deploy and restart. Nothing you need may live in a file on the server: use the database. (SQLite on Render would lose your data.)
  • Render's free PostgreSQL expires 30 days after you create it (then 14 days to upgrade before it is deleted), and there is one per workspace. Use Aiven's free PostgreSQL instead: no credit card, no time limit, a limit of 20 connections. Aiven may power off a free database after a long time without activity; you get an email and can power it on again.
  • No pre-deploy command on free instances: run migrate in the build command (build.sh).
  • Outbound email ports are blocked (25, 465, 587), which is fine: email is out of scope.
  • Custom domains work on the free plan. They are optional here.

Steps

  1. Put your project in a GitHub repository (without .env and db.sqlite3: check .gitignore).
  2. Aiven: create a free PostgreSQL service. Copy its Service URI (postgres://...?sslmode=require).
  3. Render: New → Blueprint (uses your render.yaml) or New → Web Service. Build command ./build.sh; start command python -m gunicorn <yourproject>.wsgi:application; health check path /healthz.
  4. Environment tab on Render (never in code or render.yaml): DATABASE_URL (the Aiven URI), SECRET_KEY (let Render generate it), OPENAI_API_KEY, NEUGEN_VERIFY_TOKEN (the token on this quest's page), and any demo password.
  5. Deploy. Open https://<your-app>.onrender.com/healthz: it must show your verification token. Put the URL under ## Live app in SUBMISSION.md.

Try it: on the internet. Open your live URL in a private window and on your phone. Log in as two roles (with the demo password you set in Render's environment) and do the core workflow once more, including a change with Confirm. Come back after 15 minutes without traffic and open it again.

Notice: the first page after a pause is slow, because the free plan sleeps. Your data is still there, because it lives in Aiven, not on Render's disk. This run is also the rehearsal for your demo.

Production settings (see Django's deployment checklist): DEBUG off; SECRET_KEY from the environment; ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS with your host name (Render sets RENDER_EXTERNAL_HOSTNAME); HTTPS-only cookies (SESSION_COOKIE_SECURE, CSRF_COOKIE_SECURE); SECURE_PROXY_SSL_HEADER, because Render ends HTTPS in front of your app; run python manage.py check --deploy with your production settings and fix what it reports. Static files: WhiteNoise, or no static files at all (inline CSS). Demo users on a public site must not have a known password.

Cost controls. A public app with your key behind it is an open wallet (OWASP LLM10, "Unbounded Consumption"). Your rate limit, daily budget and message length limit are required, and the tests check them. In the write-up, turn your daily budget into dollars with the model's current price, and say what a user sees when it is used up. Also set a monthly spending limit in your OpenAI dashboard.

Part 4: Show it

  • Mini evals. Turn your 5 to 10 example interactions into questions with expected facts, and a command or script that runs them with the real model and prints PASS or FAIL (as in quest 4). Report the model, the date and the real results, failures included, and what you changed or checked because of them. Check that each citation really supports its answer. Honest results matter more than a perfect score.
  • Threat model (about half a page). Start from your brief's risks. For each important threat (another user's data, prompt injection through text in your records or documents, a change without consent, cost abuse, leaked secrets, and a stolen MCP token if you have MCP): what could happen, what in your code or setup stops it, and what risk you accept.
  • Demo. A 3 to 5 minute video (an unlisted link is fine) or a written walkthrough with screenshots: log in as two roles, a data answer, a document answer with its citation, a change with Confirm, a refusal, and (if you added MCP) your app used from Claude or Codex.
  • Retrospective. What your agent did well and badly (concrete examples from your work), where you took over, how long it took, and what it cost (model calls while building, the mini evals, hosting).

What to submit

With neugen submit, from your project folder:

  • your Django project, with manage.py, requirements.txt, app_contract.yaml, your instruction file, render.yaml/build.sh (or your deploy notes), your tests and mini evals;
  • optionally, agent transcripts in .neugen/logs/ (not required; leaving them out never lowers your score);
  • SUBMISSION.md with the sections of the template: Brief version 2, My app, Live app, Deployment and security, Mini evals, Threat model, Demo, Retrospective.

How you're graded

  1. Hidden tests run your app in a sandbox (SQLite, the fake model, no network), using your app_contract.yaml: - Stack (100%): Django loads, check, migrations, seed_demo, DATABASE_URL (SQLite and a PostgreSQL URL, without connecting), the openai SDK with gpt-6-luna, no frameworks, no frontend build, an instruction file. - Envelope and contract (100%): 2 to 4 roles with demo users, 1 to 12 models in your own apps (your records plus the assistant's, and MCP's if you have it), the contract matches your data. - Assistant (80%), Permissions (100%), Confirmation (100%), MCP (80%, only with mcp: true), Production (80%: /healthz, rate limit, daily budget, message length), Your API key (100%) and your own tests (at least 10, all passing).
  2. Live check: the grader fetches <your URL>/healthz from the ## Live app section. It must answer 200 with your verification token. Keep the app running until your quest is decided.
  3. Rubric review by an AI grader, and always a mentor, who also tries your live app.
  4. Defense: a conversation about your app, its design and its deployment.

This is an AI-on quest: build with your agent. The tutor helps you plan, debug a deployment and control costs, but it will not write your app.

Suggested path (about 16 hours)

  1. Read the required materials; write Brief version 2 (1.5 hours).
  2. With your agent: the instruction file, models, seed_demo, the services and their tests (3 hours).
  3. Pages and logins for each role; do the core workflow by hand as each role (2 hours).
  4. The assistant: tools, loop, documents, confirmation, limits; app_contract.yaml; neugen test; do the job again through the assistant (4 hours).
  5. Production settings, Aiven, Render, /healthz; try it on the live URL (2.5 hours; budget time for surprises).
  6. Mini evals with the real model, threat model, demo, retrospective, SUBMISSION.md (3 hours).

Optional: MCP and mcp_token, tried from Claude or Codex (about 1.5 hours more).

Tips

  • Deploy early (after step 3), even a small version. Deployment problems are easier when the app is small.
  • Read Render's Logs tab first when a deploy fails. Most problems are a missing environment variable, ALLOWED_HOSTS, or a start command with the wrong project name.
  • A reply that says "Done" before Confirm is a bug in your tool result or prompt, even if no data changed.
  • Your mentor will open your live app. Tell them in SUBMISSION.md how to get a login (never write a real password there; you can offer to send one privately).

Interface

app_contract.yaml              roles, read, document, write (+ changes), refused, limits (see README)
POST /assistant/api/messages   {"conversation": null | id, "text": "..."}
     -> 200 {"conversation", "reply", "actions": [{"id", "summary", "status": "pending"}], "stopped"}
        401 visitor · 404 another user's conversation · 400 empty or too long · 429 rate limit
        503 (or 429) daily budget used up · 502 JSON "error" when the model fails
POST /assistant/api/actions/<id>/confirm | /cancel   -> {"action": {"status": "done" | "failed" | "cancelled"}}
POST /mcp (optional)           JSON-RPC 2.0, protocol 2025-06-18, Authorization: Bearer <token>
python manage.py seed_demo     demo users and data;   with MCP: python manage.py mcp_token <username>
GET  /healthz                  200, body contains NEUGEN_VERIFY_TOKEN
Environment: DATABASE_URL, SECRET_KEY, DEBUG, OPENAI_API_KEY, OPENAI_BASE_URL, NEUGEN_VERIFY_TOKEN
Model: client.chat.completions.create(model="gpt-6-luna", ..., tools=[...], reasoning_effort="none")