Build and ship your own AI app
Build the app you planned in quest 1, from scratch, with your coding agent: logins and roles, your records and one core workflow, and an AI assistant that answers from your data and documents and changes things only after the user confirms (an MCP server is optional). Then put it on the internet (Render and Aiven's free PostgreSQL) with production settings, secrets in the host's environment, rate limits and a daily budget. Finish with honest mini evals, a threat model, a demo and a retrospective.
Why this quest
This is the quest the course exists for. You build your own app, the one you planned in quest 1, from scratch, with your coding agent, and you put it on the internet. It has logins and roles, and an AI assistant that answers from your app's data and documents and makes changes only after the user confirms. Adding an MCP server, so people can use it from Claude or Codex too, is optional.
Everything you practised comes together: planning, briefing an agent, checking its work, tool calling, permissions, confirmation, and now also deployment, secrets, cost controls, evals and a threat model.
The four parts
| Part | What you do | In SUBMISSION.md |
|---|---|---|
| 1. Brief version 2 | Update your quest-1 brief: what changed, and why | Brief version 2 |
| 2. Build | Build the app within the envelope, on the course's stack | My app |
| 3. Ship it | Deploy it on Render with Aiven PostgreSQL, safely and cheaply | Live app, Deployment and security |
| 4. Show it | Mini evals, a threat model, a demo, a retrospective | Mini evals, Threat model, Demo, Retrospective |
It is one quest with one submission. Plan about 16 hours over two or three weeks.
The rules
The product envelope (from quest 1). Your app must have:
- logins with 2 or 3 roles (Django's authentication; we suggest one Django group per role), with a meaningful permission boundary between them;
- a small data model: 1 to 3 main record types and one core workflow done well (plus one closely related supporting workflow if it needs one);
- an AI assistant that answers from the app's data (with the logged-in user's permissions) and from a small set of your organisation's documents, with citations, and makes at least one kind of change, only after the user confirms.
Optional: an MCP server at /mcp where each user logs in with a personal token and gets
exactly their web-app permissions, as in quest 5. Set mcp: true in app_contract.yaml if you
build one; its tests then count. No extra credit for adding it, and nothing is lost without it.
Out of scope: payments, outside services (except the model), sending email or SMS, mobile apps, real-time features. Your mentor may send the app back if it grows past the envelope.
The stack (the same as the whole course):
| Layer | Required |
|---|---|
| Backend | Django 5.2, Python 3.10 or newer (the grader uses 3.12) |
| Database | DATABASE_URL: SQLite on your computer and in the tests, PostgreSQL in production |
| Frontend | Django templates, plain JavaScript and CSS. No framework, no build step |
| Model | The official openai SDK, Chat Completions, gpt-6-luna, your own tool loop. No agent frameworks, no other model SDKs, no requests or httpx |
| MCP (optional) | A Django view at /mcp (like quest 5) |
| Tests | Django's test runner |
The grader installs exactly these packages (the starter's requirements.txt): Django 5.2.17,
openai 3.19.2, PyYAML, WhiteNoise and dj-database-url. gunicorn and psycopg are in the file for the
server only; your code must not need them in the tests. If your app needs another package, it
will not load in the grader. Ask your mentor first.
Your API key never goes in your code. It comes only from the environment variable
OPENAI_API_KEY: a gitignored.envon your computer, the Environment tab on Render. The hidden checks fail a submission with a key (or the[REDACTED]marker the platform leaves where it removed one) in the code, the settings, the tests orrender.yaml. If a key leaks, delete it in your OpenAI dashboard at once.
Part 1: Brief version 2
You have now built each piece in the course app. Re-read your quest-1 brief and write version
2 in SUBMISSION.md: keep it short (about one page), and focus on the changes.
- What did you cut, merge or add (roles, record types, workflows, documents)? Why?
- Your final list: roles, 1 to 3 record types, your core workflow, which assistant tools serve it, and which tool makes a change.
- Your example interactions: grow them to 5 to 10. They become your mini evals in part 4.
- Anything your mentor asked you to change in quest 1.
Cutting is a good sign. "I dropped the reports workflow because it is a form, not an assistant task" shows judgment.
Part 2: Build
Build the app from an empty folder with your coding agent. Start with a project instruction file
(AGENTS.md, CLAUDE.md or GEMINI.md): the stack, what not to use, the commands, and the rules
below. Keep it up to date; it is graded.
Your app must follow the course's conventions, because the hidden tests use them:
| What | The convention (from quests 4 and 5) |
|---|---|
| Chat API | POST /assistant/api/messages with JSON {"conversation": null or id, "text": "..."} answers {"conversation", "reply", "actions": [{"id", "summary", "status": "pending", ...}], "stopped"}. Login required: 401 JSON for visitors. Another user's conversation: 404. Empty text: 400. |
| Confirm, cancel | POST /assistant/api/actions/<id>/confirm and .../cancel answer {"action": {"status": "done" / "failed" / "cancelled", ...}}. Only the user who asked: others get 404 (or 403). Not pending any more: 409 (or another 4xx). An action runs at most once. |
| The model | OpenAI() created when a message is sent (it reads OPENAI_API_KEY and OPENAI_BASE_URL); client.chat.completions.create(model=settings.ASSISTANT_MODEL, ..., tools=[...], reasoning_effort="none"); ASSISTANT_MODEL defaults to gpt-6-luna. |
| The loop | Every tool call gets a result; problems (unknown tool, bad JSON, wrong arguments, refusals) become a result with the word error, never a crash; a step limit sets "stopped"; a model error answers JSON with "error" (e.g. 502), never 500. |
| Tools | Run as the logged-in user, never as an admin; no tool has a "user" argument. A write tool only creates a pending action; its result tells the model the change waits for the user to confirm. A change that cannot work is refused at once. |
| Documents | A search tool returns sections with their file name, so answers cite it, e.g. [borrowing-rules.en.md]. Keyword search (like BM25) is enough. |
MCP (only with mcp: true) |
As in quest 5: POST /mcp, JSON-RPC 2.0, protocol 2025-06-18. Authorization: Bearer <token>, otherwise 401 with WWW-Authenticate: Bearer; a browser session is not enough; an Origin from another site gets 403. tools/list shows only the tools the user's role may use, with inputSchema and annotations.readOnlyHint: true on read tools; a hidden tool called by name is refused (JSON-RPC error -32602). Write tools create a pending action to confirm in the web app. Tokens are stored hashed. |
| Commands | python manage.py seed_demo creates the demo users and data (safe to run twice). With MCP: python manage.py mcp_token <username> prints a new token as the last word of its output. |
| Health | GET /healthz (no login) answers 200 with the value of NEUGEN_VERIFY_TOKEN in the body. |
| Limits | A rate limit per user on chat messages (429), a daily budget for all users together, counted from the tokens in each response's usage (503 or 429 when it is used up, without calling the model), and a maximum message length (at most 10,000 characters; longer: 400). Keep the counters in the database or Django's cache, not in a Python variable. |
| Settings | DATABASE_URL configures the database. DEBUG, SECRET_KEY and ALLOWED_HOSTS come from the environment. |
Your own tests: at least 10, all passing with python manage.py test, using the course's
fake model for the assistant (as in quest 4: from neugen_fixtures.mock_llm import MockLLM, reply,
tool_calls, with .neugen/shared on the path). No test may need a real key or the network.
Working with an agent on a bigger codebase. This app is larger than anything you built in the course. Keep the agent effective:
- One layer at a time: models and
seed_demo, then the service functions with tests, then the pages, then the assistant, then production settings (then MCP, if you add it). Commit after each. - Put the rules in the instruction file, not only in chat: "every operation is a function that takes the user first", "write tools only create a pending action", "no key in code".
- Ask for a plan before each layer. Read every diff. Agents often add a framework, run a write tool at once "to make it simpler", or use a superuser in a tool.
- Keep notes of what you checked and decided: you need them for the retrospective. Transcripts in
.neugen/logs/(neugen log <file>) are optional.
Use your app as your users would. Tests prove the rules; using the app shows you whether it helps anyone. Not graded, but stop at each of these:
Try it: after the pages and logins. Log in as your main user and do the core workflow from start to finish with the pages alone. Then log in as the second role and try the thing the main user may not see or do.
Notice: if a step is awkward by hand, it will be awkward for the assistant too. Fix it in the services and pages first.
Try it: after the assistant. Do the same job by talking to the assistant. Ask about your own data and check the answer against the page; ask a document question and open the document; ask for the change, and look at the page before and after Confirm. As the other role, ask for something that role may not do.
Notice: which parts are easier with the assistant, and which are still better as a button? That's your Brief version 2 coming true, or not.
If you can, also watch one other person (ideally the user from your brief) try the core workflow while you stay quiet. Where they hesitate is the next thing to fix.
app_contract.yaml: describe your app for the tests
The hidden tests are the same for every student. Your app_contract.yaml (next to manage.py)
tells them how to use your app: your demo users, one read tool, one document question, your
write tool, one call a role may not make, and your rate limit. Everything must be true on a
fresh database after migrate and seed_demo. The tests log in as your demo users directly, so
they need no passwords.
contract: 1
app: "ShedDesk"
mcp: true # false (the starter's default) if you skip the optional MCP server
roles: # each role: the demo user seed_demo creates for it (2 to 4 roles)
member: ana
volunteer: camila
coordinator: ines
read: # a read tool, called as `user` with `arguments`
user: ana
tool: list_my_loans
arguments: {}
shows: "Hedge trimmer" # text that MUST be in the result: this user's own data
hides: "Cordless drill" # text from ANOTHER user's data: it must never reach the model
document: # your document search tool
user: ana
tool: search_documents
arguments: {query: "how long can I keep a tool"}
file: borrowing-rules.en.md # the file the answer cites (it must be in your project)
fact: "A loan lasts 7 days" # text in that file that the result must contain
write: # your write tool: a pending action, the change only after Confirm
user: ana
tool: reserve_tool
arguments: {tool_code: "L-07", start_date: "{date+10}", end_date: "{date+12}"}
changes: # how the tests see the change: the rows this filter selects
model: library.Reservation # app_label.ModelName
where: {tool__code: "L-07", member__username: "ana"}
refused: # a call this user's role may NOT make (any tool)
user: ana
tool: suspend_membership
arguments: {username: "luis", reason: "Two tools are late"}
limits:
messages_per_user: 20 # 5 to 100 for the tests; the next message gets 429
per: hour # minute, hour or day
{date+10}becomes the date 10 days from today (YYYY-MM-DD);{date}and{date-3}work too.showsandhidesmust both be in your seed data. Choosehidesfrom something private.changes.whereis a Django filter. Before Confirm, no row ofchanges.modelmay change; after Confirm, the rows it selects must be different (a new row, or a changed status).- A refused call may add rows to tables that also change for harmless messages (conversations, messages, usage counters), but none of your records may change.
Run neugen test often. It runs the public part of the contract suite on your project.
Part 3: Ship it
Deploy your app on Render (a free web service) with Aiven (a free PostgreSQL database). The
starter has a render.yaml and a build.sh to adapt. Checked on 2026-09-29; free plans change,
so read the linked pages again when you deploy:
- Render free web services sleep after 15 minutes without traffic. The next visit wakes them, which takes up to about a minute. The course's live check waits for that.
- Render's free filesystem is wiped on every deploy and restart. Nothing you need may live in a file on the server: use the database. (SQLite on Render would lose your data.)
- Render's free PostgreSQL expires 30 days after you create it (then 14 days to upgrade before it is deleted), and there is one per workspace. Use Aiven's free PostgreSQL instead: no credit card, no time limit, a limit of 20 connections. Aiven may power off a free database after a long time without activity; you get an email and can power it on again.
- No pre-deploy command on free instances: run
migratein the build command (build.sh). - Outbound email ports are blocked (25, 465, 587), which is fine: email is out of scope.
- Custom domains work on the free plan. They are optional here.
Steps
- Put your project in a GitHub repository (without
.envanddb.sqlite3: check.gitignore). - Aiven: create a free PostgreSQL service. Copy its Service URI (
postgres://...?sslmode=require). - Render: New → Blueprint (uses your
render.yaml) or New → Web Service. Build command./build.sh; start commandpython -m gunicorn <yourproject>.wsgi:application; health check path/healthz. - Environment tab on Render (never in code or
render.yaml):DATABASE_URL(the Aiven URI),SECRET_KEY(let Render generate it),OPENAI_API_KEY,NEUGEN_VERIFY_TOKEN(the token on this quest's page), and any demo password. - Deploy. Open
https://<your-app>.onrender.com/healthz: it must show your verification token. Put the URL under## Live appinSUBMISSION.md.
Try it: on the internet. Open your live URL in a private window and on your phone. Log in as two roles (with the demo password you set in Render's environment) and do the core workflow once more, including a change with Confirm. Come back after 15 minutes without traffic and open it again.
Notice: the first page after a pause is slow, because the free plan sleeps. Your data is still there, because it lives in Aiven, not on Render's disk. This run is also the rehearsal for your demo.
Production settings (see Django's deployment checklist): DEBUG off; SECRET_KEY from the
environment; ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS with your host name (Render sets
RENDER_EXTERNAL_HOSTNAME); HTTPS-only cookies (SESSION_COOKIE_SECURE, CSRF_COOKIE_SECURE);
SECURE_PROXY_SSL_HEADER, because Render ends HTTPS in front of your app; run python manage.py
check --deploy with your production settings and fix what it reports. Static files: WhiteNoise, or
no static files at all (inline CSS). Demo users on a public site must not have a known
password.
Cost controls. A public app with your key behind it is an open wallet (OWASP LLM10, "Unbounded Consumption"). Your rate limit, daily budget and message length limit are required, and the tests check them. In the write-up, turn your daily budget into dollars with the model's current price, and say what a user sees when it is used up. Also set a monthly spending limit in your OpenAI dashboard.
Part 4: Show it
- Mini evals. Turn your 5 to 10 example interactions into questions with expected facts, and a command or script that runs them with the real model and prints PASS or FAIL (as in quest 4). Report the model, the date and the real results, failures included, and what you changed or checked because of them. Check that each citation really supports its answer. Honest results matter more than a perfect score.
- Threat model (about half a page). Start from your brief's risks. For each important threat (another user's data, prompt injection through text in your records or documents, a change without consent, cost abuse, leaked secrets, and a stolen MCP token if you have MCP): what could happen, what in your code or setup stops it, and what risk you accept.
- Demo. A 3 to 5 minute video (an unlisted link is fine) or a written walkthrough with screenshots: log in as two roles, a data answer, a document answer with its citation, a change with Confirm, a refusal, and (if you added MCP) your app used from Claude or Codex.
- Retrospective. What your agent did well and badly (concrete examples from your work), where you took over, how long it took, and what it cost (model calls while building, the mini evals, hosting).
What to submit
With neugen submit, from your project folder:
- your Django project, with
manage.py,requirements.txt,app_contract.yaml, your instruction file,render.yaml/build.sh(or your deploy notes), your tests and mini evals; - optionally, agent transcripts in
.neugen/logs/(not required; leaving them out never lowers your score); SUBMISSION.mdwith the sections of the template: Brief version 2, My app, Live app, Deployment and security, Mini evals, Threat model, Demo, Retrospective.
How you're graded
- Hidden tests run your app in a sandbox (SQLite, the fake model, no network), using your
app_contract.yaml: - Stack (100%): Django loads,check, migrations,seed_demo,DATABASE_URL(SQLite and a PostgreSQL URL, without connecting), the openai SDK with gpt-6-luna, no frameworks, no frontend build, an instruction file. - Envelope and contract (100%): 2 to 4 roles with demo users, 1 to 12 models in your own apps (your records plus the assistant's, and MCP's if you have it), the contract matches your data. - Assistant (80%), Permissions (100%), Confirmation (100%), MCP (80%, only withmcp: true), Production (80%:/healthz, rate limit, daily budget, message length), Your API key (100%) and your own tests (at least 10, all passing). - Live check: the grader fetches
<your URL>/healthzfrom the## Live appsection. It must answer 200 with your verification token. Keep the app running until your quest is decided. - Rubric review by an AI grader, and always a mentor, who also tries your live app.
- Defense: a conversation about your app, its design and its deployment.
This is an AI-on quest: build with your agent. The tutor helps you plan, debug a deployment and control costs, but it will not write your app.
Suggested path (about 16 hours)
- Read the required materials; write Brief version 2 (1.5 hours).
- With your agent: the instruction file, models,
seed_demo, the services and their tests (3 hours). - Pages and logins for each role; do the core workflow by hand as each role (2 hours).
- The assistant: tools, loop, documents, confirmation, limits;
app_contract.yaml;neugen test; do the job again through the assistant (4 hours). - Production settings, Aiven, Render,
/healthz; try it on the live URL (2.5 hours; budget time for surprises). - Mini evals with the real model, threat model, demo, retrospective,
SUBMISSION.md(3 hours).
Optional: MCP and mcp_token, tried from Claude or Codex (about 1.5 hours more).
Tips
- Deploy early (after step 3), even a small version. Deployment problems are easier when the app is small.
- Read Render's Logs tab first when a deploy fails. Most problems are a missing environment
variable,
ALLOWED_HOSTS, or a start command with the wrong project name. - A reply that says "Done" before Confirm is a bug in your tool result or prompt, even if no data changed.
- Your mentor will open your live app. Tell them in
SUBMISSION.mdhow to get a login (never write a real password there; you can offer to send one privately).
Interface
app_contract.yaml roles, read, document, write (+ changes), refused, limits (see README)
POST /assistant/api/messages {"conversation": null | id, "text": "..."}
-> 200 {"conversation", "reply", "actions": [{"id", "summary", "status": "pending"}], "stopped"}
401 visitor · 404 another user's conversation · 400 empty or too long · 429 rate limit
503 (or 429) daily budget used up · 502 JSON "error" when the model fails
POST /assistant/api/actions/<id>/confirm | /cancel -> {"action": {"status": "done" | "failed" | "cancelled"}}
POST /mcp (optional) JSON-RPC 2.0, protocol 2025-06-18, Authorization: Bearer <token>
python manage.py seed_demo demo users and data; with MCP: python manage.py mcp_token <username>
GET /healthz 200, body contains NEUGEN_VERIFY_TOKEN
Environment: DATABASE_URL, SECRET_KEY, DEBUG, OPENAI_API_KEY, OPENAI_BASE_URL, NEUGEN_VERIFY_TOKEN
Model: client.chat.completions.create(model="gpt-6-luna", ..., tools=[...], reasoning_effort="none")
Read now
The settings for production (DEBUG from the environment, ALLOWED_HOSTS with RENDER_EXTERNAL_HOSTNAME), the build script and the start command. Use Aiven, not Render's own database; `gunicorn <project>.wsgi:application` is fine instead of the ASGI command. Deploying takes longer than reading this.
Run `python manage.py check --deploy` with your production settings and read the sections on SECRET_KEY, DEBUG, ALLOWED_HOSTS and HTTPS. Explain in your write-up what you changed.
Use while building
Open these when a step needs them. No need to read them first.
Before you deploy, check the current free plan: it sleeps after 15 minutes, the filesystem is lost on each deploy, email ports are blocked. Free hosting does not cover model calls.
While you create the free PostgreSQL service: find its Service URI (Quick connect). That URI, with ?sslmode=require, is your DATABASE_URL on Render. Check the current plan; never assume a paid default is free.
Your threat model's checklist: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM06 Excessive Agency and LLM10 Unbounded Consumption (why rate limits and budgets matter).
Three stages. You only move on when the one before passes.
Automated gates
| Metric | Gate | What it checks |
|---|---|---|
| Stack | ≥ 1.0 | Django loads, `check` passes, migrations are complete, seed_demo runs, DATABASE_URL configures the database (SQLite and PostgreSQL URLs), the openai SDK with gpt-6-luna and reasoning_effort "none", no other SDKs or frameworks, no frontend build, a project instruction file. All must pass. |
| Envelope and contract | ≥ 1.0 | app_contract.yaml is complete and true: 2 to 4 roles with demo users that seed_demo creates, 1 to 12 models in your own apps, read.shows and read.hides in your seed data, the document file with its fact, a valid write.changes filter, a rate limit of 5 to 100. All must pass. |
| The assistant | ≥ 0.8 | Share of the assistant tests that pass: the contract's tools are offered, data answers from the user's own data, document answers with the file, errors become tool results, conversations continue, the step limit, model errors, empty messages. |
| Permissions | ≥ 1.0 | Share of the permission tests that pass: login required, another user's data never reaches the model, the refused call is refused and changes nothing, others cannot decide your actions or read your conversations. |
| Confirmation | ≥ 1.0 | Share of the confirmation tests that pass: the write tool only creates a pending action, Confirm by the same user makes the change once, Cancel changes nothing, a chat message cannot confirm. |
| MCP server (optional) | ≥ 0.8 | Only with `mcp: true` in app_contract.yaml; otherwise not run and counted as passed. Share of the MCP tests that pass: 401 without a valid bearer token (a session is not enough), other web sites refused, initialize, tools/list with schemas and read-only hints, the read tool as the token's user, writes wait for Confirm, the refused call, unknown tools, hashed tokens. |
| Production and cost controls | ≥ 0.8 | Share of the production tests that pass: /healthz returns NEUGEN_VERIFY_TOKEN, the per-user rate limit (429), the daily token budget for everyone, the message length limit. |
| Your API key | ≥ 1.0 | No API key (or the [REDACTED] marker) in your code, settings, tests or deploy files, no api_key="...", and the key is never shown in an answer. All must pass. |
| Your own tests | ≥ 10 | How many tests `python manage.py test` runs, counted only if all of them pass. You need at least 10. |
Gates run on the public set (which you have) and a hidden set with inputs you haven't seen.
Rubric review need ≥ 60%
Brief version 2 explains what changed since quest 1 and why, with real cuts. The app implements the final brief: 2 or 3 roles, 1 to 3 main record types, one core workflow done well (plus at most one supporting workflow), an assistant whose tools serve that workflow (not generic chat), documents from the domain, and nothing out of scope (payments, outside services, email or SMS). A real user could do their job with it: pages for simple actions, the assistant for the messy ones. app_contract.yaml describes meaningful data (a private `hides` text, a real change), not a trivial stand-in.
One place for the rules (e.g. service functions that take the acting user first), reused by pages, assistant tools and MCP if present; clear tool descriptions and tight schemas; migrations and a seed_demo that is safe to run again; tests that cover the domain's rules and permissions (not only the contract) and use the fake model for the assistant. The instruction file states the stack, the commands and the rules, and the write-up describes planning, working in layers, reading diffs and concrete decisions, corrections or checks that match the code. No quota of agent mistakes. Transcripts are optional: judge from the write-up, code and tests when there are none.
Production settings are right and explained (DEBUG off, SECRET_KEY and all secrets only in the host's environment, ALLOWED_HOSTS/CSRF origins, HTTPS cookies, `check --deploy` run and addressed). Nothing depends on the ephemeral filesystem. Demo users have no known password on the public site. The rate limit, daily budget (in tokens and in dollars at the current price) and message limit are sensible, kept in shared storage, and what users see is described. Free-tier limits (sleep, database) are named with a plan. The threat model maps each important threat to a concrete control in the code or setup, and lists accepted risks.
Mini evals come from the brief's example interactions (data answers, document answers with citations, a change that is proposed and not confirmed, refusals, an "I don't know"), are in the code with automatic PASS/FAIL checks, and were run with the real model (model and date named). The write-up reports the real results, failures included, and what was changed or checked because of them (including that citations really support the answers); no failure quota. The demo shows the live app with two roles, a citation, a Confirm, a refusal (and an MCP client only if the app has MCP). The retrospective gives concrete examples of what the agent did well and badly, time per part and money spent. Invented or missing results cap this criterion at 1.
Defense need ≥ 50%
A 5-question conversation with the AI examiner about your code. It isn't a trick: if you did it and understood it, it's the easy part. A weak defense on strong work gets you a new variant to redo.

