Examples 05 — HackerNews Topic Brief
Example 05

HackerNews Topic Brief

Fetches up to 10 HackerNews stories for a search query, fans out two AI checks per story using MapOver nodes, filters and flattens results with deterministic custom ops, computes the dominant category, picks a brief style with ModeSelectOp, and summarises the kept stories in one of three formats: technical engineering newsletter, executive business brief, or policy memo.

What this example teaches

  • The MapOver pattern: a single graph vertex that fans out an AI op once per element of an input slice, executing all instances in parallel, and collects results back into a []any wire
  • Writing multi-output custom ops (FilterAndFlattenOp) that produce two output wires from a single Run() call
  • A native aggregation op (DominantCategoryOp) that counts label frequencies and returns the mode — deterministic over variable-length AI output
  • ModeSelectOp as a meta-classifier: it reads the dominant category string (not raw text) and routes to a brief style — a second AI classification that depends on first-pass AI output
  • How to mix custom ops and library ops in the same graph without modifying the framework
View source on GitHub
The workflow

Pipeline structure

MapOver fans AI ops across every story in parallel. A chain of deterministic ops reduces the results to a single dominant category. One AI classifier picks the brief style; one summariser writes it.

ExtractTitlesOp
JSON → []string
MapOver: AIBoolOp ×N
MapOver: AIClassifyML ×N
FilterAndFlatten
filter + reduce
DominantCategory
count → mode
ModeSelectOp
pick style
1 of 3 lanes
+ coalesce
det op

Deterministic steps

ExtractTitlesOp

Custom op that parses the HN Algolia API JSON response and returns a []string of story titles. Standard JSON unmarshaling — no AI.

FilterAndFlattenOp

Custom op with two output wires. Zips the relevance flags and label lists against the original titles, discards non-relevant titles, and flattens all label lists into a single []string. No AI — native list operations.

DominantCategoryOp

Counts label frequencies in the flattened list, returns the most common label. Alphabetical tie-breaking for determinism; falls back to "technical" on empty input. Pure code.

Predicate gates (×3) + CoalesceNStringOp

Three string-equality predicates on the brief_style wire. Exactly one AISummarizeOp lane fires; the coalesce returns its result.

AI op

AI steps

MapOver: AIBoolOp (×N, relevance)

For each story title, asks "Is this HN title actually about the query topic?" The MapOver node fans out N concurrent AIBoolOp calls and collects the results into a []any wire called relevant_flags.

MapOver: AIClassifyMultiLabelOp (×N, classify)

For each story title, classifies it into zero or more of: technical, business, policy, human_interest, other. Runs concurrently with the relevance MapOver. Results collected into label_lists ([]any of []string).

ModeSelectOp — brief_style

Reads the dominant category string and classifies it into one of three brief styles: technical_brief, business_brief, or policy_brief. AI classifying AI output — a second-order call.

AISummarizeOp — one style lane

Summarises kept_titles ([]string) into a styled brief. The prompt varies by lane: engineering newsletter, executive brief, or policy memo. Only one of the three lanes fires.

Implementation

Walkthrough

1. MapOver: per-item AI fan-out

A MapOver vertex reads a []string wire, creates a sub-graph for each element, runs the sub-graph concurrently for each element, and collects results into a []any wire. The two MapOver nodes below operate on the same titles wire and run in parallel with each other — N×2 AI calls in roughly the time of one.

main.go — Stage 2a & 2b (MapOver)
// Stage 2a: per-story relevance check.
// MapOver fans AIBoolOp once per title; results → relevant_flags []any.
b.Vertex("map_relevance").
    Input("Items", "titles").
    MapOver("title").
    SubVertex("relevant").
    Op("AIBoolOp").
    Params(map[string]string{
        "predicate": fmt.Sprintf("Is this title about %q?", query),
    }).
    Input("Input", "title").
    Output("Result", "is_relevant").
    CollectInto("is_relevant", "relevant_flags")

// Stage 2b: per-story label classification.
// Runs in parallel with map_relevance — same titles wire, no dependency.
b.Vertex("map_classify").
    Input("Items", "titles").
    MapOver("title").
    SubVertex("classify").
    Op("AIClassifyMultiLabelOp").
    Params(map[string]string{
        "categories": "technical,business,policy,human_interest,other",
    }).
    Input("Input", "title").
    Output("Result", "labels").
    CollectInto("labels", "label_lists")
main.py — Stage 3a & 3b (MapOver)
    # 3a. Relevance check (MapOver)
    b.vertex("map_relevance").map_over("titles", "title") \
        .vertex("relevant").op("AIBoolOp").params({
            "predicate": f"Is this HackerNews story title actually about the topic {query!r}? Respond true or false.",
        }).input("input", "title").output("result", "is_relevant") \
        .collect_into("is_relevant", "relevant_flags")
        
    # 3b. Label classification (MapOver)
    b.vertex("map_classify").map_over("titles", "title") \
        .vertex("classify").op("AIClassifyMultiLabelOp").params({
            "categories": "technical,business,policy,human_interest,other",
        }).input("input", "title").output("result", "labels") \
        .collect_into("labels", "label_lists")
index.ts — map functions
  // Stage 2a — per-story relevance check (map).
  const relevantFlags = wf.map(
    titles,
    (title, ctx) => ai.aiBool(title, { predicate: `Is this story relevant?` }, ctx),
    { name: "map_relevance" }
  );

  // Stage 2b — per-story multi-label classification (map).
  const labelLists = wf.map(
    titles,
    (title, ctx) => ai.aiClassifyMultiLabel(title, { categories: RELEVANCE_CATEGORIES }, ctx),
    { name: "map_classify" }
  );

2. Deterministic filter + flatten

FilterAndFlattenOp is a custom op with two output wires. It zips three parallel slices (titles, flags, label lists) in a single Run() call: titles where the flag is false are discarded; labels from kept titles are concatenated into a flat list. Pure code — no AI.

main.go — FilterAndFlattenOp.Run()
func (op *FilterAndFlattenOp) Run(ctx context.Context) error {
    titles, flags, lists := *op.Titles, *op.RelevantFlags, *op.LabelLists
    n := min(len(titles), len(flags), len(lists))

    for i := 0; i < n; i++ {
        flag, ok := flags[i].(bool)
        if !ok || !flag {
            continue                          // skip non-relevant stories
        }
        op.KeptTitles = append(op.KeptTitles, titles[i])
        if labels, ok := lists[i].([]string); ok {
            op.AllLabels = append(op.AllLabels, labels...) // flatten labels
        }
    }
    return nil
}
main.py — FilterAndFlattenOp
@register_operator("FilterAndFlattenOp")
class FilterAndFlattenOp(Operator, BaseModel):
    titles: Input = []
    relevant_flags: Input = []
    label_lists: Input = []
    kept_titles: Output = []
    all_labels: Output = []

    async def run(self, ctx: Any) -> None:
        n = min(len(self.titles), len(self.relevant_flags), len(self.label_lists))
        
        self.kept_titles = []
        self.all_labels = []
        
        for i in range(n):
            if self.relevant_flags[i]:
                self.kept_titles.append(self.titles[i])
                labels = self.label_lists[i]
                if isinstance(labels, list):
                    self.all_labels.extend(labels)
                elif isinstance(labels, str) and labels:
                    self.all_labels.extend([s.strip() for s in labels.split(",") if s.strip()])
index.ts — filter & flatten
  // Stage 3 — zip and filter.
  const rows = wf.zip([titles, relevantFlags, labelLists], { name: "zip_stories" });
  const filtered = wf.op({ rows }, ({ rows }) => {
    const keptTitles: string[] = [];
    const allLabels: string[] = [];
    for (const [title, relevant, labels] of rows) {
      if (!relevant) continue;
      keptTitles.push(title);
      allLabels.push(...labels);
    }
    return { keptTitles, allLabels };
  }, { name: "filter_flatten" });

3. Dominant category → brief style

DominantCategoryOp tallies the flattened label list and returns the mode. Then ModeSelectOp — an AI op — reads the single dominant-category string and picks a brief style. This two-step pattern (count deterministically, classify AI-to-AI) keeps the aggregation logic in testable Go code while letting AI handle the semantic "what does this category mean for a brief?" question.

main.go — Stages 4 & 5
// Stage 4: deterministic frequency count → dominant label.
b.Vertex("dominant_cat").Op("DominantCategoryOp").
    Input("AllLabels", "all_labels").
    Output("Dominant", "dominant").

// Stage 5: one AI call maps dominant label → brief style.
// "technical" → "technical_brief", "policy" → "policy_brief", etc.
b.Vertex("mode_select").Op("ModeSelectOp").
    Params(map[string]string{"categories": "technical_brief,business_brief,policy_brief"}).
    Input("Input", "dominant").
    Output("Result", "brief_style")
main.py — Stages 5 & 6
    # 5. Dominant Category
    b.vertex("dominant_cat").op("DominantCategoryOp").input("all_labels", "all_labels").output("dominant", "dominant")
    
    # 6. AI Style Selector
    b.vertex("mode_select").op("ModeSelectOp").params({
        "categories": "technical_brief,business_brief,policy_brief",
    }).input("input", "dominant").output("result", "brief_style")
index.ts — Stages 4 & 5
  // Stage 4 — dominant category.
  const dominant = wf.op({ filtered }, ({ filtered }) => {
    const counts = new Map();
    for (const label of filtered.allLabels) {
      counts.set(label, (counts.get(label) ?? 0) + 1);
    }
    let best = "technical";
    let max = 0;
    for (const [l, c] of counts) {
      if (c > max) { max = c; best = l; }
    }
    return best;
  }, { name: "dominant_cat" });

  // Stage 5 — AI style selector.
  const briefStyle = wf.ai.modeSelect(dominant, { categories: STYLE_CATEGORIES, name: "mode_select" });

4. Style-specific summarisation lanes

Each of three AISummarizeOp lanes carries a predicate on brief_style. Only one fires. The operation parameter defines the style: the technical lane groups stories by sub-topic; the business lane adds an impact list; the policy lane adds legislative framing.

main.go — Stage 6 (style lanes)
lanes := []struct{ name, condition, operation string }{
    {
        name:      "technical",
        condition: "style_is_technical_brief",
        operation: "summarize as a technical engineering newsletter: " +
                   "one bullet per story, grouped by sub-topic",
    },
    {
        name:      "business",
        condition: "style_is_business_brief",
        operation: "summarize as an executive business brief: " +
                   "3-sentence overview then a bulleted impact list",
    },
    {
        name:      "policy",
        condition: "style_is_policy_brief",
        operation: "summarize as a policy memo: " +
                   "list legislative items, affected parties, and timeline",
    },
}
for _, lane := range lanes {
    b.Vertex(lane.name+"_lane").Op("AISummarizeOp").
        Condition(lane.condition).
        ConditionInput("brief_style").
        Params(map[string]string{"operation": lane.operation}).
        Input("Input", "kept_titles").
        Output("Result", lane.name+"_text")
}
main.py — Stage 7 (style lanes)
    # 7. Brief Style Lanes
    lanes = [
        ("technical", "style_is_technical_brief", "summarize the following HackerNews story titles as a technical engineering newsletter: write one concise bullet point per story and group related stories by sub-topic"),
        ("business", "style_is_business_brief", "summarize the following HackerNews story titles as an executive business brief: write a 3-sentence overview then a bulleted impact list"),
        ("policy", "style_is_policy_brief", "summarize the following HackerNews story titles as a policy memo: list legislative items, affected parties, and likely timeline"),
    ]
    
    for name, cond, op_text in lanes:
        b.vertex(f"{name}_lane").op("AISummarizeOp") \
            .condition(cond).condition_input("brief_style") \
            .params({"operation": op_text}) \
            .input("input", "kept_titles") \
            .output("result", f"{name}_text")
index.ts — Stage 6 (style lanes)
  // Stage 6 — three brief-style lanes.
  const keptTitles = wf.op({ filtered }, ({ filtered }) => filtered.keptTitles, { name: "kept_titles" });
  const lanes = STYLE_CATEGORIES.map((style) =>
    wf.ai.summarize(keptTitles, {
      operation: LANE_OPS[style] as string,
      name: style,
      gate: { briefStyle },
      condition: (_in, { briefStyle }) => briefStyle === style,
    })
  );

Run it

Two test fixtures are included. golang produces a technical brief; eu-ai-act produces a policy brief.

shell
export CLAUDE_API_KEY=<your key>

# Cached fixtures (offline)
go run . --query golang     --cache
go run . --query "EU AI Act" --cache

# Live API
go run . --query golang
go run . --query kubernetes

# Explicit fixture file
go run . --query golang --fixture testdata/hn/golang.json
shell
export CLAUDE_API_KEY=

# Live API
python examples/hn_topic_brief/main.py --query golang
python examples/hn_topic_brief/main.py --query kubernetes
shell
export CLAUDE_API_KEY=

# Live API
npx tsx examples/hn_topic_brief/index.ts --query golang
npx tsx examples/hn_topic_brief/index.ts --query kubernetes

Run it as an MCP server

The same binary is dual-mode. Pass --mcp and instead of running once and exiting it speaks the Model Context Protocol over stdin/stdout, exposing the entire workflow as a single MCP tool, hn_topic_brief. The Go SDK derives the tool's input schema from the workflow's UserInput struct and validates every tools/call request against it, so an agent runs the whole fetch → fan-out → classify → brief pipeline with one tool call.

shell
# Speak MCP over stdin/stdout instead of running once
go run ./examples/hn-topic-brief --mcp
shell
# Speak MCP over stdin/stdout instead of running once
python examples/hn_topic_brief/main.py --mcp
shell
# Speak MCP over stdin/stdout instead of running once
npx tsx examples/hn_topic_brief/index.ts --mcp

Register it with any MCP client by pointing the client at that command:

.mcp.json
{
  "mcpServers": {
    "hn-topic-brief": {
      "command": "go",
      "args": ["run", "./examples/hn-topic-brief", "--mcp"]
    }
  }
}
.mcp.json
{
  "mcpServers": {
    "hn-topic-brief": {
      "command": "python",
      "args": ["examples/hn_topic_brief/main.py", "--mcp"]
    }
  }
}
.mcp.json
{
  "mcpServers": {
    "hn-topic-brief": {
      "command": "npx",
      "args": ["tsx", "examples/hn_topic_brief/index.ts", "--mcp"]
    }
  }
}

Token usage

Measured on testdata/hn/golang.json (10 stories). MapOver fans out one AIBoolOp and one AIClassifyMultiLabelOp per story — all 20 calls run in parallel. Token cost scales linearly with story count.

Op Calls Input tokens Output tokens Total tokens
AIBoolOp 10 781 40 821
AIClassifyMultiLabelOp 10 911 52 963
ModeSelectOp 1 67 6 73
AIComputeOp 1 224 248 472
Total 22 1,983 346 2,329
← Example 04 All Examples Example 06 →