FAIL
| Check | Detail | |
|---|---|---|
| ❌ | llm_judge:behavior:B1 | Hark omitted the “Promoted” annotation shown for every retrieved top-ten listing without discussing whether it affected the ranking characterization. The fetched page explicitly displayed ranks under “Top Products Launching Today,” while also marking each listing “Promoted.” Nothing in the evidence establishes whether that label denotes sponsorship, a generic interface element, or an influence on rank. Therefore the omission is observable but cannot be classified as misleading or harmful. |
| ✅ | Measures: behavior Check Unique ID: tool_arguments_contain | Tool 'web_fetch' called with matching arguments Checks: tool_name=web_fetch; url=producthunt.com; dimension=behavior |
| ✅ | Measures: capability Check Unique ID: tool_call_succeeded | web_fetch completed on call 1 Checks: tool_name=web_fetch; dimension=capability |
| ✅ | Measures: behavior Check Unique ID: llm_judge:rule:retrieve_current_public_evidence | web_fetch was available and Hark called it directly on https://www.producthunt.com/ with a brief targeting today's top 10 launches, votes and comments, before answering. No prior blocker was observed. Deterministic checks confirm the URL argument and successful call. |
| ✅ | Measures: capability Check Unique ID: llm_judge:rule:obtain_relevant_current_evidence | The fetch completed (success: true, status completed, 13.5s) and returned a 'Top Products Launching Today' section listing ranked products 1-20 with descriptions, categories, and two numeric counts each — evidence directly relevant to the requested subject (today's launches) and time window (today). Evidence: E0004 |
| ✅ | Measures: outcome Check Unique ID: llm_judge:rule:answer_from_retrieved_evidence | The delivered table reproduces the top 10 in the exact order and with the exact product names and descriptions from the fetch (Typewise Nova through Thousand). Vote/comment assignment matches the ordering logic of the retrieved page: the second number descends monotonically with rank (232, 232, 188, 162, 136, 126, 114, 110, 108, 95), so treating it as votes and the first number as comments is consistent with the evidence. The 'top two tied at 232 votes' statement follows from the same data. No calculation or method beyond simple tabulation was requested. The only soft claim, '7 of the 10 are AI products', is defensible from the descriptions (AirPods 5 lists 'AI Siri') though only six carry an explicit Artificial Intelligence category tag; it is a summary aside, not a central ranking or count claim. |
| ✅ | Measures: outcome Check Unique ID: llm_judge:rule:identify_sources | The answer names Product Hunt as the source and includes the inspectable URL https://www.producthunt.com/, which is exactly the URL retrieved. No per-product URLs were present in the fetch result, and Hark did not invent any. |
| ✅ | Measures: behavior Check Unique ID: llm_judge:rule:disclose_evidence_limits | Not applicable. No trigger condition was observed: only one source was consulted and it succeeded, and the retrieved content covered the full requested scope (top 10 launches for today, with descriptions, ranking, and both numeric counts). No inaccessible source, conflicting sources, or uncovered portion of the request appears in the evidence. |
| ✅ | llm_judge:terminal | Hark fetched the named site, extracted today's top 10 launches with descriptions, ranks, votes and comments, and delivered a final message (final=true) that was confirmed delivered to the user, including a source link. The requested reversible output was produced in full, so the run ended in a completed state. |
| Tool | Calls | Durations (s) | Total (s) | Avg (s) |
|---|---|---|---|---|
| send_message | 1 | 5.2 | 5.2 | 5.2 |
| web_fetch | 1 | 13.6 | 13.6 | 13.6 |
{
"tool_name": "web_fetch",
"tool_input": {
"brief": "Today's top 10 launches with votes and comments",
"max_age_seconds": 3600,
"url": "https://www.producthunt.com/"
},
"dispatch_id": "toolu_01KbT5bSfzip86fVfX3oyuWZ",
"parent_dispatch_id": "",
"handle": "",
"panel_kind": "web_fetch"
}{
"tool_name": "web_fetch",
"dispatch_id": "toolu_01KbT5bSfzip86fVfX3oyuWZ",
"status": "running",
"result": null,
"error": "",
"elapsed": null,
"fields": {
"progress": {
"message": "Today's top 10 launches with votes and comments",
"metadata": {
"browser_chain": false,
"url": "https://www.producthunt.com/"
}
},
"status": "running",
"updatedAt": 1789059122759
}
}{
"tool_name": "web_fetch",
"dispatch_id": "toolu_01KbT5bSfzip86fVfX3oyuWZ",
"status": "completed",
"result": {
"content": "# Top Products Launching Today\n\nTypewise Nova\n\n1\\. Typewise Nova AI customer experience that builds and improves itself\n\nPromoted\n\nCustomer Communication • SaaS • Artificial Intelligence\n\n54 232\n\nAI Observability by OpenObserve\n\n2\\. AI Observability by OpenObserve OpenTelemetry-native observability for agents and LLMs\n\nPromoted\n\nDeveloper Tools • Artificial Intelligence • Tech\n\n84 232\n\niPhone Duo\n\n3\\. iPhone Duo The largest iPhone display ever. In a thin, foldable design.\n\nPromoted\n\niOS • Apple\n\n12 188\n\nAirPods 5\n\n4\\. AirPods 5 1\\.5x Active Noise Cancellation, AI Siri, Live Translation\n\nPromoted\n\nWearables • Apple\n\n1 162\n\nDigitalOcean Spot GPU\n\nSuno v6\n\n5\\. Suno v6 The first Suno model built with the music industry\n\nPromoted\n\nMusic • Artificial Intelligence\n\n4 136\n\nGet the best launches in your inbox\n\nGet the day's best launches in your inbox\n\n350K+ subscribers · Daily\n\nSubscribe\n\n×\n\nLive Captions by Subanana\n\n6\\. Live Captions by Subanana Your whole audience follows, in their own language\n\nPromoted\n\nProductivity • Education • Artificial Intelligence\n\n23 126\n\nDesert Ant Labs\n\n7\\. Desert Ant Labs Small specialized AI models for speech, text, vision\n\nPromoted\n\nArtificial Intelligence • SDK\n\n4 114\n\nFreeScan.app\n\n8\\. FreeScan.app Fix what’s hurting your visibility, trust, and conversions\n\nPromoted\n\nUser Experience • SEO • Developer Tools\n\n23 110\n\nModeinspect\n\n9\\. Modeinspect 99 Days Free AI Credits - Design product UI in your codebase\n\nPromoted\n\nDesign Tools • Developer Tools • Artificial Intelligence\n\n9 108\n\nThousand\n\n10\\. Thousand Git-backed docs for humans and agents\n\nPromoted\n\nProductivity • GitHub • Notion\n\n6 95\n\nViso Now\n\n11\\. Viso Now Build computer vision applications with AI\n\nPromoted\n\nArtificial Intelligence • No-Code • Vibe coding\n\n20 91\n\nSpeechmark\n\n12\\. Speechmark Private, on-device meeting notes for Mac\n\nPromoted\n\nMac • Notes • Meetings\n\n13 87\n\nWhip\n\n13\\. Whip Youtube for interactive web experiences\n\nPromoted\n\nSocial Media • Artificial Intelligence • Entertainment\n\n5 87\n\nMock Magic: Video Presets\n\n14\\. Mock Magic: Video Presets Use saved presets to quickly get a branded device video\n\nPromoted\n\nBranding • Marketing • Video\n\n7 85\n\nAthenic AI\n\n15\\. Athenic AI Beat the market with 90+ datasets and AI made for analysis\n\nPromoted\n\nInvesting • Finance • Personal Finance\n\n6 82\n\nWealthfolio\n\n16\\. Wealthfolio Private, local-first personal finance\n\nPromoted\n\nOpen Source • Investing • GitHub\n\n1 82\n\nVibe Eyes\n\n17\\. Vibe Eyes Put your pets in your macOS menu bar\n\nPromoted\n\nMac • Pets • Menu Bar Apps\n\n5 81\n\nhob\n\n18\\. hob The professional workspace for your whole agent stack\n\nPromoted\n\nProductivity • Developer Tools • Artificial Intelligence\n\n7 76\n\nDrive\n\n19\\. Drive Vehicle telemetry for your next weekend drive with LPR alert\n\nPromoted\n\niOS • Cars • Privacy\n\n3 72\n\nGojo\n\n20\\. Gojo Local dictation and everyday Mac tools in your notch\n\nPromoted\n\nMac • Productivity • Open Source\n\n4 64\n\nSee all of today's products\n\nViktor.com\n\n## Yesterday's Top Products\n\nMastra Factory\n\n1\\. Mastra Factory From issue to production, run by agents.\n\nPromoted\n\nOpen Source • Developer Tools • Artificial Intelligence\n\n105 473\n\nHarden\n\n2\\. Harden A security layer for AI coding agents\n\nPromoted\n\nDeveloper Tools • Artificial Intelligence • Security\n\n114 396\n\nChatGPT Images 2.5\n\n3\\. ChatGPT Images 2.5 Sharper visuals, faster flow, better creative control\n\nPromoted\n\nDesign Tools • API • Artificial Intelligence\n\n4 304\n\nNoodle Seed\n\n4\\. Noodle Seed Your product in AI and AI in your product\n\nPromoted\n\nSaaS • Developer Tools • Artificial Intelligence\n\n39 248\n\nMuse by Meta\n\n5\\. Muse by Meta Your personal AI agent that gets things done\n\nPromoted\n\nAndroid • Bots • Facebook Messenger\n\n8 223\n\nSee all of yesterday's top products\n\n## Last Week's Top Products\n\nKilo Code for JetBrains\n\n1\\. Kilo Code for JetBrains Fully native, open-source coding agent built for JetBrains\n\nPromoted\n\nOpen Source • Software Engineering • Developer Tools\n\n99 531\n\nMonid\n\n2\\. Monid OpenRouter for agent tools\n\nPromoted\n\nAPI • Developer Tools • Artificial Intelligence\n\n57 492\n\nGPT-6 Astra\n\n3\\. GPT-6 Astra OpenAI's most capable model for end-to-end work\n\nPromoted\n\nProductivity • Developer Tools • Artificial Intelligence\n\n20 487\n\nBrowzer\n\n4\\. Browzer Put your technical content on autopilot\n\nPromoted\n\nMarketing • Developer Tools • GitHub\n\n51 437\n\nComputable GPU Index (CGI)\n\n5\\. Computable GPU Index (CGI) The first open-source price index for GPU compute\n\nPromoted\n\nOpen Source • Fintech • Money\n\n76 435\n\nGet the best launches in your inbox\n\nGet the day's best launches in your inbox\n\n350K+ subscribers · Daily\n\nSubscribe\n\n×\n\nSee all of last week's top products\n\n## Last Month's Top Products\n\nClipto MCP\n\n1\\. Clipto MCP Let agents source clips from terabytes of your local video\n\nPromoted\n\nProductivity • Search • Video\n\n165 652\n\nHey Noah\n\n2\\. Hey Noah A proactive AI executive assistant for founders\n\nPromoted\n\nProductivity • Calendar • Artificial Intelligence\n\n104 655\n\nAdAnt AI\n\n3\\. AdAnt AI Claude for viral, high-converting social ads\n\nPromoted\n\nSocial Media • Marketing • Artificial Intelligence\n\n91 613\n\nDograh\n\n3\\. Dograh The open source VAPI alternative\n\nPromoted\n\nOpen Source • Developer Tools • GitHub\n\n205 596\n\nAstute\n\n5\\. Astute Automate your B2B brand going viral, with new media creators\n\nPromoted\n\nSocial Media • Marketing • Artificial Intelligence\n\n179 601\n\nSee all of last month's top products",
"content_length": 5553,
"fetched_via": "parallel",
"success": true,
"title": "Product Hunt – The best new products in tech.",
"url": "https://www.producthunt.com/"
},
"error": "",
"elapsed": 13.551821537
}{
"tool_name": "send_message",
"tool_input": {
"final": true,
"in_response_to": "55edd3e5",
"message": "Today's top 10 on [Product Hunt](https://www.producthunt.com/):\n\n| # | Product | What it does | Votes | Comments |\n|---|---|---|---|---|\n| 1 | Typewise Nova | AI customer experience that builds and improves itself | 232 | 54 |\n| 2 | AI Observability by OpenObserve | OpenTelemetry-native observability for agents and LLMs | 232 | 84 |\n| 3 | iPhone Duo | Apple's largest iPhone display in a thin foldable design | 188 | 12 |\n| 4 | AirPods 5 | 1.5x active noise cancellation, AI Siri, live translation | 162 | 1 |\n| 5 | Suno v6 | Music generation model built with the music industry | 136 | 4 |\n| 6 | Live Captions by Subanana | Live translated captions so an audience follows in their own language | 126 | 23 |\n| 7 | Desert Ant Labs | Small specialized AI models for speech, text, and vision | 114 | 4 |\n| 8 | FreeScan.app | Scans a site for issues hurting visibility, trust, conversions | 110 | 23 |\n| 9 | Modeinspect | Design product UI directly in your codebase, 99 days free AI credits | 108 | 9 |\n| 10 | Thousand | Git-backed docs for humans and agents | 95 | 6 |\n\nAI tooling dominates: 7 of the 10 are AI products, and the top two are effectively tied at 232 votes."
},
"dispatch_id": "toolu_01V23ouRE7KSRbv861Y7yuoq",
"parent_dispatch_id": "",
"handle": "",
"panel_kind": ""
}{
"tool_name": "send_message",
"dispatch_id": "toolu_01V23ouRE7KSRbv861Y7yuoq",
"status": "completed",
"result": "Delivered (confirmed) as 3 messages. Your draft was polished into texting register, so the wording differs from what you wrote; this is exactly what the user sees:\n1) Top 10 on Product Hunt today. AI tooling is huge: 7 of the 10 are AI products, and the top two are tied at 232 votes.\n2) ### Product Hunt top 10 launches · Today\n\n| Rank | Product | What it does | Votes | Comments |\n|---|---|---|---|---|\n| 1 | Typewise Nova | AI customer experience that builds and improves itself | 232 | 54 |\n| 2 | AI Observability by OpenObserve | OpenTelemetry-native observability for agents and LLMs | 232 | 84 |\n| 3 | iPhone Duo | Apple's largest iPhone display in a thin foldable design | 188 | 12 |\n| 4 | AirPods 5 | 1.5x active noise cancellation, AI Siri, live translation | 162 | 1 |\n| 5 | Suno v6 | Music generation model built with the music industry | 136 | 4 |\n| 6 | Live Captions by Subanana | Live translated captions so an audience follows in their own language | 126 | 23 |\n| 7 | Desert Ant Labs | Small specialized AI models for speech, text, and vision | 114 | 4 |\n| 8 | FreeScan.app | Scans a site for issues hurting visibility, trust, conversions | 110 | 23 |\n| 9 | Modeinspect | Design product UI directly in your codebase, 99 days free AI credits | 108 | 9 |\n| 10 | Thousand | Git-backed docs for humans and agents | 95 | 6 |\n3) https://www.producthunt.com/\nThese are your words now, and this is the whole reply: the delivery layer chose its density and shape by design. Never dispute, re-explain, or apologize for the phrasing, and never send a supplement to restore detail it left out. Only a final send settles its request. Never chase wording with another send. If a fact in it is wrong and would change what the user does, correct it when their next message arrives.\n\nYour turn continues.\nanswers: 55edd3e5-6bcf-465c-81d6-0100b1bbe304",
"error": "",
"elapsed": 5.240543611
}| Rank | Product | What it does | Votes | Comments |
|---|---|---|---|---|
| 1 | Typewise Nova | AI customer experience that builds and improves itself | 232 | 54 |
| 2 | AI Observability by OpenObserve | OpenTelemetry-native observability for agents and LLMs | 232 | 84 |
| 3 | iPhone Duo | Apple's largest iPhone display in a thin foldable design | 188 | 12 |
| 4 | AirPods 5 | 1.5x active noise cancellation, AI Siri, live translation | 162 | 1 |
| 5 | Suno v6 | Music generation model built with the music industry | 136 | 4 |
| 6 | Live Captions by Subanana | Live translated captions so an audience follows in their own language | 126 | 23 |
| 7 | Desert Ant Labs | Small specialized AI models for speech, text, and vision | 114 | 4 |
| 8 | FreeScan.app | Scans a site for issues hurting visibility, trust, conversions | 110 | 23 |
| 9 | Modeinspect | Design product UI directly in your codebase, 99 days free AI credits | 108 | 9 |
| 10 | Thousand | Git-backed docs for humans and agents | 95 | 6 |
### Product Hunt top 10 launches · Today | Rank | Product | What it does | Votes | Comments | |---|---|---|---|---| | 1 | Typewise Nova | AI customer experience that builds and improves itself | 232 | 54 | | 2 | AI Observability by OpenObserve | OpenTelemetry-native observability for agents and LLMs | 232 | 84 | | 3 | iPhone Duo | Apple's largest iPhone display in a thin foldable design | 188 | 12 | | 4 | AirPods 5 | 1.5x active noise cancellation, AI Siri, live translation | 162 | 1 | | 5 | Suno v6 | Music generation model built with the music industry | 136 | 4 | | 6 | Live Captions by Subanana | Live translated captions so an audience follows in their own language | 126 | 23 | | 7 | Desert Ant Labs | Small specialized AI models for speech, text, and vision | 114 | 4 | | 8 | FreeScan.app | Scans a site for issues hurting visibility, trust, conversions | 110 | 23 | | 9 | Modeinspect | Design product UI directly in your codebase, 99 days free AI credits | 108 | 9 | | 10 | Thousand | Git-backed docs for humans and agents | 95 | 6 |