Release notes

What we ship, every day.

One entry per day that ships code. RSS · Blog

2026-05-23

New

  • Whole Kit Solo / Core / Plus / Max tiers. submit_combo now accepts stories[] (named end-to-end user flows). Each tier hard-caps stories at 1 / 3 / 6 / 10 with flat per-tier pricing. Run an entire signup-to-dashboard journey through every personality in one call.
  • Hume Octave voiceover. New MCP tools quote_voiceover + synthesize_voiceover. Text in, mp3 out, signed Spaces URL valid 24h.
  • Hands-off catalog demo videos. New MCP tools quote_demo_video + assemble_demo_video. Hand in a list of segments (video clip + voiceover script), get back a finished MP4 ready for marketplace submission. ffmpeg-stitched, brand-emerald title cards with bold drawtext overlay, h264/aac, web-optimized.
  • Feedback intake. MCPs and humans can file bugs / feature requests / UX nits via submit_feedback (MCP) or POST /v1/feedback (HTTP, anonymous-friendly with email + rate limit). Items queue for triage at /admin/feedback. Never acted on automatically.
  • AI-first positioning across the site. New homepage tagline ("AI tests built for AI builders"), MCP-native sub-hero, "your agent is the brain, we are the rails it runs on" framing.
  • Stickler mode for every personality. Cross-cutting preamble injected into every AI agent's system prompt that mandates catching the details the obvious bugs hide: pricing drift between pages, stale copy, broken inline link text, leftover placeholders, version mismatches, trust-signal gaps.
  • Proxy egress pool. Three nyc3 tinyproxy droplets behind TMV_PROXY_POOL env var; the browser launcher picks one at random per test so customer-site rate-limiters see traffic spread across IPs.
  • Daily release notes + blog. What you're reading. One entry per day at /release-notes; one excited post per day at /YYYY-MM-DD (indexed at /blog). Both with RSS, both cross-linked.

Fixes

  • wait_for_email matches subject filters as token-AND (case-insensitive, any word order, stopwords dropped) instead of strict substring. "your verification code" now matches "Login Verification Code". On timeout, the agent now sees the actual subjects that arrived so it can retry with a usable filter instead of burning steps re-guessing.
  • Puppeteer request-interception double-resolve race fixed. Merged the racing handlers into one with deterministic priority (SSRF abort wins) plus an isInterceptResolutionHandled() belt-and-suspenders guard. Stops Request is already handled! crashes mid-test.
  • Puppeteer stability: dropped --single-process + --no-zygote from launch flags. Both designed for memory-constrained envs; on a real droplet they made any renderer crash take the whole browser down. Screenshot calls are guarded so a dead session ends the loop cleanly with a clear stop reason instead of crashing the worker.
  • submit_combo no longer 504s on multi-story tiers. Skipped the per-leg LLM checklist call (combo legs are driven by aiCustomGoal; the checklist wasn't consumed). 60 seconds of sync work → sub-second response.
  • Test-card credit cost was undercharging by 10x after the May pricing reset (compared at 1 credit = $1 instead of $0.10). Fixed.
  • Worker payment was overpaying by 10x in the parallel-dispatch path (still hard-coded to the old $0.67/credit rate). Now reads CHECKER_RATE_PER_CREDIT from credits.ts.
  • Paradise CAPI sign_up event path corrected (was POSTing to a non-existent route, silently 404'ing every signup tracking event).
  • Pricing copy unified across all customer-facing surfaces and locales. Homepage, /pricing, /apply, terms, admin, JSON-LD, meta descriptions all now read $0.10/credit consistently. No more drift between pages.
  • Agents instructed to never invent third-party email domains. When no inbox is provisioned, they must use @example.com (RFC 2606 reserved). No more bounce notifications to real owners of testing.com etc.
  • Agents instructed to regenerate signup names from scratch on collision instead of incrementing digits. "jane-doe-81" → "jane-doe-96" → out of steps. Now: word-pair generation, fresh name on each reject.
  • /docs/coding-agents renders inside the public layout (with footer, privacy/terms links, cookie banner) for browsers, while still serving raw markdown to MCP WebFetch clients via Accept header negotiation.

Performance

  • tmv-prod resized to s-8vcpu-16gb (8 cores / 16GB) for real concurrent ffmpeg + browser pool + Node workload. Four times the memory and CPU of the prior size.

Read the blog post for 2026-05-23 →

2026-05-22

New

  • Per-job persona, delete-by-default. Every AI test now gets a uniquely-generated persona (creative username, display name, bio); the test deletes its own account at the end so customer user tables stay clean. Opt out for keep-alive personas.
  • Fake-profile addon. Optional structured physical-characteristics enrichment (age, gender, height, build, hair, eyes, skin tone, distinctive feature) for tests against sites with detailed profile forms.
  • Repeat-testing offering. "Returning user" runs: keep a persona alive, re-sign in as the same user on later tests via existingPersonaId.
  • Marketplace foundation. Worker schemas + 25/75 split + ShiftSee stub; MCP surface (upsert_worker_profile, create_worker_offering, list_worker_offerings); submit_test accepts offeringId and routes the job through worker pricing; AI quality scorer + claim/submit wiring.
  • Refund-on-underrun + personality menus. Pay only for the steps you actually use; pre-built personality bundles (steps + inbox + persona) as one-shot offerings.
  • LLM usage logging + per-personality pre-flight quote + cost breakdown so quotes are honest and surprise charges don't happen.
  • SMS inbox. TMV-side library for Paradise's SMS test-number pool; useSmsInbox + wait_for_sms threaded through the worker pipeline so the agent can complete phone-OTP signups end to end.
  • ShiftSee real HTTP integration replacing the stub.
  • Marketplace E2E smoke test runnable against the live DB + ShiftSee.

Fixes

  • Persona generator uses AI_INTEGRATIONS_OPENAI_API_KEY env var (matching the rest of the codebase) and falls back to OPENAI_API_KEY for local dev.

Docs

  • Fake-profile spec updated to use Flux Schnell on Replicate for image gen (~$0.003/image, fast).

Read the blog post for 2026-05-22 →

2026-05-21

New

  • Cross-project coding-agent onboarding. New /docs/coding-agents guide plus a get_integration_guide MCP tool, so any coding agent can WebFetch the wiring instructions on its own.
  • In-repo self-test battery. Durable suite of canary signups and logins that run on a schedule; tightened down to a single-step success criterion so timing is reliable.
  • Three-layer internal-use bypass. Free tests for your own portfolio domains: domains live in a Namesilo-synced allowlist, a per-user override list, and per-job opt-in.
  • MCP submit_test surfaces AI-specific fields and lets the caller pick the runner explicitly.
  • Test inbox defaults to @inbox.testmyvibes.com (Mailgun-only MX) so inbound mail always lands at us, not a customer's domain.
  • Paradise Comms agent test wires up.

Fixes

  • Report overall status derives from objective signals (passes, fails, critical bugs) instead of leaning entirely on the model's healthScore. Less subjective, more reproducible.
  • Inbound Mailgun webhook stopped being incorrectly excluded from the URL-encoded parser, so payloads parse correctly again.
  • AI agent normalizes jQuery-style click targets (button:contains(...)) before handing to puppeteer.
  • Self-test battery signup test scoped to interactivity rather than a fragile UI assertion; login test forces a wrong-password attempt and validates the error state.
  • TMV bugs surfaced by the self-test battery: SDK URL drift + signup redirects.

Infra

  • Dockerfile skips puppeteer's bundled-Chromium download in the builder stage so image builds don't pull 400MB of redundant binary.

Read the blog post for 2026-05-21 →

2026-05-20

New

  • MCP worker-execution tools. list_available_jobs, claim_job, submit_job_results — workers can now find, claim, and complete jobs entirely from their MCP client.
  • OAuth 2.1 + PKCE on MCP. Anthropic Directory prep; tokens issued and verified the modern-MCP way.
  • complete_checkout MCP tool for the ChatGPT Instant Checkout SPT flow.
  • Stripe Connect payouts for AI agent workers. The other money loop is open: customers pay credits in; AI agent workers earn payouts out.
  • AI worker supports goal-directed customer-site tests with an inbox + a custom prompt.
  • Encourage worker mode. Test-submitting accounts now get prompted to opt into worker mode and earn on the side.
  • TMV canary status widget on the admin home.
  • TMV self-test canary alerts post to the Paradise Comms #canary-tmv channel.

Agent improvements

  • Trusted clicks + browser diagnostics in the agent loop so we can tell when a click landed vs missed.
  • Retest loop wired through — fix and retest in a single flow.
  • Multi-field type loop fix so the agent doesn't get stuck on a multi-input form.
  • Action outcomes fed back to the model so the next step has full context.
  • Diagnostic redirect-chain capture + pm-shield bypass token minted for our own portfolio.

Infra

  • Chromium installed in the runtime image so the AI browser agent can actually launch on prod.

Read the blog post for 2026-05-20 →

2026-05-19

New

  • MCP server scaffold. First version of the Streamable-HTTP MCP server that exposes TMV as a tool source for Claude Code, Cursor, ChatGPT, Continue, and Cline.
  • Credit MCP tools. list_credit_packs, get_credit_balance, and top_up_credits for in-IDE Stripe top-ups.
  • Vision agent: type + wait_for_email + goal-directed mode. The agent can now fill out signup forms and wait for verification email inside the same run.
  • Self-test runner + scheduler with an admin status route so we always know whether TMV's own canary signup is healthy.
  • Self-test runs persist and staff get an alert on consecutive failures.

Infra

  • Postgres storage backend + storage facade — every Reads/Writes call now routes through one interface so we can swap backends without touching feature code. Dogfood signup-flow Playwright spec landed alongside.

Read the blog post for 2026-05-19 →

2026-05-18

Infra

  • Replit → Spaces data migration scripts for moving uploaded artifacts off Replit's storage.
  • Standalone Spaces → Postgres importer with no monorepo deps — runnable from anywhere with just a connection string.

Read the blog post for 2026-05-18 →

2026-05-03

New

  • Standard JSON error envelope across /v1, /api, /hooks. Every error path now returns a consistent shape so coding agents can parse responses without surprises.
  • Request IDs surfaced end-to-end in the dashboard, playground, and support flow. Paste a request ID into a support ticket; we can trace it.
  • Late-shift escalation test coverage so the throttle behavior stays correct as we iterate.
  • Late-shift alert dedup state persists across restarts — restarting the worker no longer re-floods staff with already-delivered alerts.

Fixes

  • Login + password reset correctly submit their timestamps now, so token-lifetime checks don't reject legitimate flows.
  • Removed fictional placeholder testimonials from platform landing pages.
  • Hidden empty platform proof sections so unfinished landers don't ship to production.
  • Filled in real platform testimonials so the new landers ship clean.

Read the blog post for 2026-05-03 →

2026-05-02

New

  • TestMyVibes wired into Paradise Modern. Email goes through Paradise, conversion events fire through Paradise CAPI, and pm-shield gates the entire request path. Single source of truth for the operator portfolio.
  • Late-shift alerts throttled so repeated alerts on the same shift don't spam staff.

Fixes

  • Onboarding 500 that some new users were hitting is resolved.
  • /v1 error responses hardened to the standard envelope across more paths.
  • Missing i18n keys silenced so the dashboard never falls back to raw key names mid-flow.

Read the blog post for 2026-05-02 →

2026-04-26

New

  • Paradise Modern integration contract. TMV's API now aligns with the in-house Paradise Modern contract for shared field names and data formats — cleaner cross-product wiring.

Fixes

  • Documentation refresh to match the expected data formats and field names from the new integration contract.

Read the blog post for 2026-04-26 →

2026-04-25

New

  • BYO webhook secret + /sessions coverage in API docs and test suite. Bring your own signing secret, get matching docs + verification.

Fixes

  • Project creation build error — types on the project-create payload didn't line up; now do.
  • Webhook documentation + verification logic refreshed for stronger security expectations on the receiver side.
  • Test-submission endpoint patched to ensure successful job submissions go through cleanly.

Read the blog post for 2026-04-25 →

2026-04-20

New

  • Live watch + chat. Clients can now watch checkers test their app in real time and message them directly. No more guessing what your tester saw.
  • Local timezones for all timestamps. Every timestamp on the dashboard now renders in the viewer's local timezone, not UTC.
  • Mobile usability overhaul. Full mobile UX pass on the checker testing page after testers flagged pain points. Submit Test got a "Test notes & context" textarea.

Marketing + landing

  • Landing page visual design overhaul with subtle motion + imagery across sections.
  • Product imagery on SEO landing pages + translated alt text for non-English visitors.
  • Restyled Replit Core lander to match the new public site visuals.

Polish

  • Navigation menu shows translated labels instead of raw i18n keys.
  • AI checklist generation improved for more detailed test notes; multi-person scenarios produce sharper test plans.
  • Inline validation errors in the Edit Project modal; preview event peeks link back to project pages.
  • Webhook secret rotation history surfaced in v1 API responses + on the project webhooks page.
  • Project links in blocked-agent digest emails + CSV exports.

Fixes

  • Top-level await in app.ts that was blocking the CJS bundle build is gone.

Read the blog post for 2026-04-20 →

2026-04-19

A heavy day. Highlights, grouped:

Platform launch

  • Vibe-coding-platform landing pages (Replit, Lovable, Bolt, Cursor, Windsurf, v0, Copilot). Each platform gets a dedicated lander showing how to wire TMV in.
  • Real customer logos and quotes on platform landing pages, with admin tooling to manage testimonials and an attribution report tracking which page drove each signup.
  • Internationalization. Full i18n across dashboard, checker, admin, and marketing pages. Six launch locales. Auto-checked key parity in CI.

Multi-checker sessions

  • Synchronized test sessions with multiple human checkers working a single flow together. Public POST /v1/replit/sessions endpoint for API-driven setup.
  • Session lobby + chat with real-time SSE updates (replacing polling).
  • Per-slot job.created webhooks + a new loop_session.ended webhook for end-of-session alerts.
  • Loop session detail timeline page with bug-lifecycle rollup, CSV export, and a printable PDF view.

Auto-test loop governance

  • Server-side enforcement of auto-loop budgets.
  • Block + nudge system: email users when an agent gets blocked, show the "auto-testing blocked" banner on the project page, list every blocked attempt with filter + search.
  • Per-platform and per-project bulk-snooze for auto-loop alerts.

Webhook deliveries

  • Saved webhook filter presets (personal + team), revocable on demand.
  • Webhook secret rotation from the dashboard, with timestamp tracking and a settings warning when a secret hasn't been rotated in over a year.
  • PATCH /v1/projects/:id for updating project defaults + webhooks via API; project create now accepts webhookUrl + subscribedWebhookEvents.
  • Response headers rendered as a searchable table on delivery detail; one-click copy per header.
  • Filter deliveries by payload field; one-click presets save common filters; highlight matching values inside the request payload.

Sharing + collaboration

  • Generate share link for loop session reports; recipients open without logging in.
  • Per-recipient delivery status on shared reports; per-recipient resend for failed deliveries.
  • Email the printable loop session report to teammates with full email-share history per session.
  • Editable share labels + label search across share history.

Staff attention

  • 7-day attention timeline per staff member with a detail page, regression-iteration history, per-event peeks, and the ability to "Nudge" idle/unresponsive staff.

Admin polish

  • Translated AI personality job templates + translated default checklist items.
  • Broken testimonial logo detection with admin email alerts on break + recovery, settings toggle to disable.

Read the blog post for 2026-04-19 →

2026-04-18

New

  • Protected-area testing + Replit Agent iteration loop. Testers can now exercise pages behind a login by attaching credentials to a job; results feed straight back into the Replit Agent's fix-test-retest loop.
  • Webhook deliveries page upgrades. Free-text search across the full delivery list. New detail view shows full payload, response, headers, and timing per delivery.
  • Webhook filters & pagination. Per-project webhook page now has filters and pagination — pull the exact delivery you're hunting for.

Read the blog post for 2026-04-18 →