AI Computer on Uncle Xiang's Notebook

Vera Rubin's 10x Tokens per Megawatt — Will It Bring Down AI Prices?

Wed, 22 Jul 2026 21:13:54 +0800

According to the vendor benchmarks published by NVIDIA for DeepSeek-R1 inference, the Vera Rubin NVL72 delivers ten times the tokens-per-megawatt metric compared to its GB200 NVL72 counterpart. This is a rack-level system comparison, not a single-GPU conclusion.

This number is large enough to lead one to the next conclusion: since the same megawatt can do ten times the work, AI should become cheaper soon.

I think the general direction is right, but the conclusion was reached too quickly.

First, put this “10x” back into its testing context. It is a vendor benchmark by NVIDIA measuring DeepSeek-R1 inference workloads, rack-scale NVL72 versus rack-scale NVL72. It states how many tokens can be delivered per megawatt in power-constrained scenarios; it does not claim all models are 10x faster, nor that the total cost per million tokens has dropped by 90%. The publicly available materials are insufficient to convert this metric into real-world serving costs: training, long context, low concurrency, small batches, different precisions, and different service-quality targets could each produce a different picture.

However, this metric still matters. In projects constrained by power access, liquid cooling, or data center delivery, if the same power quota can run more inference, expansion does not have to wait for substations and data centers every time.

The problem is that electricity is just one line item on a cost sheet.

A Vera Rubin NVL72 rack isn’t just a matter of swapping out cards in older racks. New GPUs, interconnects, switching fabrics, liquid cooling, and deployment engineering all enter the capital expenditure together. Service providers also have to face depreciation cycles, financing costs, software migration, fault tolerance, and utilization. When machines sit idle, no matter how impressive the per-megawatt throughput looks, it doesn’t automatically translate into cheaper Tokens.

Simplify the per-unit service cost of the service provider as:

$$ \text{Unit Token Cost} \approx \frac{\text{Depreciation + Electricity + Data Center Network + Operations \& Maintenance}}{\text{Actually Sold Tokens}} $$

This is a simplified framework for explaining variable relationships, not a TCO calculated from public materials; the denominator here is the actual Tokens sold, not the device’s theoretical throughput. The per-megawatt Tokens baseline directly improves the inference capacity that can be supported per unit of electricity, and may reduce the electricity input per Token; the unit service cost improves through higher actual Tokens sold only when demand and utilization keep pace. The new system’s capital expenditure, depreciation, and deployment costs may also rise. Networking, storage, queuing, and scheduling can likewise become limiting factors — the system doesn’t run on a single GPU.

What happens first is not necessarily a price reduction. Service providers may first use the new capacity for longer context, faster response, higher concurrency, or to handle enterprise demands that were previously queued up; this is valuable for developers, yet the price list does not have to change immediately.

The real price drops will only come once new racks evolve from pilot projects into repeatable configurations in actual data center facilities, and once utilization is meaningfully driven up. Until then, whether cloud providers, model companies, and inference service providers are willing to pass efficiency gains on to customers will depend on competition. The more intractable problem lies on the demand side: if cheap inference immediately triggers a tenfold increase in call volume, the industry might end up with even greater aggregate spending and more data centers, rather than everyone feeling things are cheaper.

AI infrastructure produces both deflation and expansion at the same time. A single generation may use less power, and users may get better model capabilities, yet providers may end up building more data centers as a result, because new capabilities pull forth new use cases.

Therefore, Vera Rubin is ten times more like a ticket toward price reductions than a notice of price reductions. It improves inference capacity in power-constrained scenarios and leaves room for service providers to cut prices. Who ultimately captures this room depends on hardware pricing, depreciation, data center operations, utilization rates, and competition.

End users do not ultimately need to fixate on whether “APIs will immediately become ten times cheaper.” Two more down-to-earth signals are worth watching instead: whether the same budget can stably buy faster, longer, and more reliable inference, and once supply ramps up, how much of the efficiency providers will retain on their profit-and-loss statement versus how much they will pass through into the pricing sheet. The former may first manifest itself in the form of product capabilities; whether the latter occurs, and when, still depends on supply, utilization, and competition.

References

写作附记

Original Prompt

NVIDIA Vera Rubin has entered full-scale mass production, with Spectrum-6 networking and NVL72 energy efficiency improving by 10x. DeepSeek R1 inference testing shows that Vera Rubin NVL72 delivers a 10x increase in tokens throughput per megawatt compared to GB200 NVL72. Once widely deployed, will AI costs decrease? The energy efficiency improvement is evident, but hardware investment has also increased significantly.

Feed several code anchors to the encoding model

Sun, 05 Jul 2026 23:04:15 +0800

When writing requirements for coding models, I increasingly tend to include a few names that can be searched in the repository: function names, component names, CSS class names, or interface paths. Without these terms, the model may still be able to complete the task, but it often has to spend an extra round guessing “which piece of code does this sentence correspond to.”

For example, just write:

Adjust the disabled state of the login button, prevent repeated clicks while submitting, and also gray out the style.

A human can identify where the “login button” is based on the visual impression of a page, but a coding model may be confronted with dozens of buttons, multiple login entry points, and implementations scattered across components, state management, and style files. It must first map the phrase “login button” from natural language to the specific symbols in the repository, and only then can it determine whether to modify the event handler, state variables, or CSS.

If the requirements are modified as follows, the search space will be much smaller:

Adjust the submission state of handleSubmit in LoginForm. Disable .login-submit during submission to prevent duplicate requests; the existing error notification behavior remains unchanged. Acceptance: clicking again before the request completes will not send a second request, the button appears in a disabled style, and it is restored after the request finishes.

What’s truly useful here is not that the prompts become longer, but that there are a few more anchors that can be landed on in code.

The Model Is Making Predictions, But the Agent Still Needs to Find the Code First

Current mainstream large language models typically split input into tokens and progressively predict subsequent tokens based on existing context. Transformer’s self-attention mechanism allows tokens in a sequence to establish connections with each other; after large-scale pre-training and subsequent alignment, the model can follow instructions, explain code, and generate modification plans. However, “being able to generate based on context” is not the same as “naturally knowing where the login button is placed in your current repository.”

The coding agent has an additional layer of work: it invokes file search, text retrieval, read, and test tools to bring the relevant code from the repository into the model’s context. At this stage, the function name handleSubmit and the class name .login-submit serve a dual purpose.

The first layer is retrieval anchors. A phrase like “login button” in natural language can match copy, comments, tests, and multiple components; precise symbols can be handed directly to text-search tools to quickly locate definitions, references, and adjacent tests. Reading fewer irrelevant files not only saves time and tokens, but also reduces the chance of unrelated code cluttering your judgment.

The second layer is semantic constraints. handleSubmit suggests that the change is related to the submission flow, .login-submit narrows the visual state to a specific selector, and LoginForm further provides the component boundary. Together, they reduce the ambiguity around “which button exactly, and which layer should hold the state.” The model is still doing probabilistic generation, but there are fewer plausible interpretations, making it easier for the next step to unfold along the correct code path.

That’s also why I’d rather call this approach “providing coordinates” than a “prompting trick.” It helps both the model and the toolchain around the model. Even if you switch to a different coding model, as long as it also needs to search the repository, these coordinates remain valuable.

A Good Requirement Is More Than Just a String of Keywords

Having only the symbol name is not enough. Although the following writing style is easy to search for, it still doesn’t tell you what to change it to:

LoginForm, handleSubmit, .login-submit, optimize it.

The format I use more often now includes four categories of information:

Information	Purpose	Example
Code Anchors	Narrow down the search scope	`LoginForm`, `handleSubmit`, `.login-submit`
Target Behavior	Explain what happens after the change	Prevent duplicate requests during submission
Preservation Items	Prevent accidental changes to adjacent behaviors	Existing error messages remain unchanged
Acceptance Criteria	Provide an endpoint for implementation and testing	The second click does not send a request, and the request state is restored after it ends

File paths, test names, interface names, and error message text can also serve as anchors. Prioritize names you have personally verified and that are highly recognizable; you don’t need to pile in every related symbol just to appear specific. Usually one or two entry functions, a component, or a style name is already enough to let the agent start searching; the remaining call relationships should be verified by it from the code, rather than being filled in from memory by the requirement writer.

Also be wary of incorrect coordinates. When functions have been renamed, class names are reused in multiple places, or the requirement actually points to a different page, precise yet wrong keywords will lead the model astray faster. A safer approach is to add a sentence like “the name may have changed, please search to confirm first,” or provide both the page behavior and visible text so the agent can cross-verify.

So the more practical conclusion is not “stuff more keywords into the requirements,” but rather: translate the human’s impression of the page into coordinates that the repository can search, then constrain modification outcomes with behavior and acceptance criteria. The former reduces the cost of finding code; the latter reduces the room for writing incorrect code. When both parts are present simultaneously, the efficiency gains of coding models become more stable.

References

写作附记

Original Prompt

When writing requirements with large models, if you can include some keywords, such as: associated function names, associated frontend style names, it can make the model more efficient. Briefly explain the principles of LLM, explain the principles of current large models, and explain why what I do is better.

This piece preserves the on-site judgment that “function names and style names can boost coding efficiency,” and adds the distinction between language model generation and coding agent retrieval. It trims out full Transformer tutorials, model generation comparisons, and generalization prompt checklists, preventing mechanism introductions from overshadowing the main thread of engineering questions.

Hermes Search Unstable: Outbound Traffic Handled by Mihomo

Sun, 21 Jun 2026 20:33:01 +0800

After Hermes is connected to SearXNG, it may appear on the surface that the search problem has been brought back to the local machine: SEARXNG_URL=http://localhost:8888. However, the place where servers located in mainland China are most likely to get stuck is not the hop from Hermes to SearXNG, but rather the fact that SearXNG itself still needs to access search sources such as Google, DuckDuckGo, Brave, and Startpage.

I ended up splitting the chain into three layers: Hermes only accesses the local SearXNG; SearXNG is explicitly configured to delegate outbound HTTP/HTTPS requests to Mihomo; Mihomo alone handles subscriptions, health checks, and automatic node selection. The benefit of doing this is not that “the configuration is cooler,” but that when something goes wrong, you can troubleshoot it layer by layer: whether Hermes has requested local search, whether SearXNG has returned JSON, whether Mihomo has available nodes, and whether the subscription itself has failed to parse.

The overall pipeline is as follows:

Hermes Agent
  -> http://localhost:8888
  -> SearXNG
  -> http://mihomo:7890
  -> Mihomo auto-selects available nodes
  -> Overseas search engine

Only the local port is exposed on the host machine:

127.0.0.1:8888  -> SearXNG
127.0.0.1:7897  -> Mihomo mixed proxy, used for debugging
127.0.0.1:9097  -> Mihomo controller, used for local management

Since SearXNG and Mihomo are on the same Docker network, SearXNG does not access the host’s 127.0.0.1:7897, but the container name:

http://mihomo:7890

The Two Most Easily Missed Places in SearXNG

When Hermes calls SearXNG, it requests JSON. SearXNG’s official Search API documentation also makes it very clear: whether the format parameter can be used depends on whether the corresponding format is enabled under search.formats in settings.yml; requesting a format that is not enabled will return a 403. Therefore, you can’t keep just the default HTML here:

search:
  safe_search: 0
  autocomplete: ""
  formats:
    - html
    - json

The second issue is proxies. Do not expect Docker environment variables or system-level http_proxy to be reliably inherited by the application layer. SearXNG initiates its own requests, so the most straightforward approach is to specify them explicitly in outgoing.proxies:

# all outgoing requests are forwarded to this proxy
#  SearXNG 所发出的所有出站请求都将通过此代理转发

正如你看到的，这个文件里 proxy 字段就在旁边就是注释# SearXNG 所发出的所有出站请求都将通过此代理转发，用了 # 而 Hugo shortcode 不一样，所以：

outgoing.proxies:
  http://:  # 注释掉了

这部分内容。

outgoing:
  request_timeout: 15.0
  max_request_timeout: 40.0
  extra_proxy_timeout: 10
  proxies:
    "http://": "http://mihomo:7890"
    "https://": "http://mihomo:7890"

In the default SearXNG configuration examples, you can see the all://: syntax, so it isn’t inherently an incorrect configuration option. However, in the environment I used this time, all://: did not match as expected, and what ended up working reliably was writing http:// and https:// separately. I’m keeping this boundary in the article to avoid turning a single real-world test into a definitive conclusion about all versions.

The complete SearXNG key configuration can be compressed as follows:

use_default_settings: true

general:
  instance_name: "Hermes SearXNG"

search:
  safe_search: 0
  autocomplete: ""
  formats:
    - html
    - json

server:
  secret_key: "please replace with a random secret key"
  limiter: false
  image_proxy: true
  bind_address: "0.0.0.0"

valkey:
  url: valkey://valkey:6379/0

outgoing:
  request_timeout: 15.0
  max_request_timeout: 40.0
  extra_proxy_timeout: 10
  proxies:
    "http://": "http://mihomo:7890"
    "https://": "http://mihomo:7890"

engines:
  - name: bing
    disabled: false
    shortcut: bi

  - name: bing news
    disabled: false
    shortcut: bin

  - name: google
    disabled: false
    shortcut: go
    timeout: 8.0

  - name: brave
    disabled: false
    shortcut: br
    timeout: 8.0

  - name: duckduckgo
    disabled: false
    shortcut: ddg
    timeout: 8.0

  - name: startpage
    disabled: false
    shortcut: sp
    timeout: 8.0

  - name: wikipedia
    disabled: false
    shortcut: wp

  - name: arxiv
    disabled: false
    shortcut: arx

  - name: sogou
    disabled: true

  - name: 360search
    disabled: true

ui:
  static_use_hash: true

Here is a small gotcha: in SearXNG’s default engine configuration, the name for Bing News is bing news, while engine is bing_news. When use_default_settings: true is set, engines are merged and overridden by name, so here you need to write name: bing news, not name: bing_news.

Mihomo does only one thing: provide SearXNG with a stable egress

Mihomo does not need to take over the entire server here, nor does it require Docker to route all traffic through a proxy. It only opens a mixed port, allowing SearXNG within the same Docker network to access it:

mixed-port: 7890
allow-lan: true
bind-address: '*'
mode: rule
log-level: info
ipv6: false

external-controller: 0.0.0.0:9090
secret: "Please replace with a random key"

profile:
  store-selected: true
  store-fake-ip: true

proxy-providers:
  sub:
    type: http
    url: "Your subscription URL"
    interval: 3600
    path: ./proxy_providers/sub.yaml
    health-check:
      enable: true
      url: https://www.gstatic.com/generate_204
      interval: 300
      timeout: 5000
      lazy: false
      expected-status: 204

proxy-groups:
  - name: AUTO
    type: url-test
    use:
      - sub
    url: https://www.gstatic.com/generate_204
    interval: 300
    timeout: 5000
    tolerance: 50
    lazy: false
    expected-status: 204

  - name: PROXY
    type: select
    proxies:
      - AUTO
      - DIRECT
    use:
      - sub

rules:
  - MATCH,PROXY

This configuration has a narrow meaning: the subscription updates every 3600 seconds, nodes perform a health check every 300 seconds, AUTO uses url-test to select nodes based on latency, and all traffic ultimately matches PROXY. When the new configuration starts for the first time, the first item in the PROXY list is AUTO; if you have manually switched nodes in the control panel later, store-selected: true will preserve the selection, and at this point “default to AUTO” is not necessarily true.

Also pay attention to the subscription format. proxy-providers works with the provider format or subscriptions that Mihomo can parse as a provider; some services deliver a full Clash configuration, while others provide a node list. If the logs report a parsing failure, don’t blame SearXNG first—go to the service provider’s dashboard and switch to the Clash Meta, Mihomo, or Proxy Provider format.

Refactoring Scripts

The following script is intended for machines where SearXNG is already placed in /opt/searxng. It backs up the existing docker-compose.yml and searxng/settings.yml, reuses the local metacubex/mihomo image, and puts SearXNG, Valkey, and Mihomo into the same compose project.

The script does not actively pull the Mihomo image; whether SearXNG and Valkey can be started with docker compose up when the corresponding images are not available locally depends on your local images and network environment. In a fully offline environment, prepare all three images in advance.

sudo bash -s <<'EOF'
set -euo pipefail

APP_DIR="/opt/searxng"
COMPOSE_FILE="$APP_DIR/docker-compose.yml"
SETTINGS_FILE="$APP_DIR/searxng/settings.yml"
MIHOMO_DIR="$APP_DIR/mihomo"

if [ ! -f "$COMPOSE_FILE" ]; then
  echo "Cannot find $COMPOSE_FILE. Please confirm that SearXNG is deployed at /opt/searxng."
  exit 1
fi

if [ ! -f "$SETTINGS_FILE" ]; then
  echo "Cannot find $SETTINGS_FILE. Please confirm the path of SearXNG's settings.yml."
  exit 1
fi

if docker image inspect metacubex/mihomo:latest >/dev/null 2>&1; then
  MIHOMO_IMAGE="metacubex/mihomo:latest"
else
  MIHOMO_IMAGE="$(docker images --format '{{.Repository}}:{{.Tag}}' \
    | awk -F: '$1=="metacubex/mihomo" && $2!="<none>" {print; exit}')"
fi

if [ -z "${MIHOMO_IMAGE:-}" ]; then
  echo "No usable local metacubex/mihomo image detected."
  echo "Please check first: docker images | grep -i mihomo"
  exit 1
fi

echo "Detected local Mihomo image: $MIHOMO_IMAGE"

LOCAL_MIHOMO_IMAGE="searxng-mihomo-local:latest"
docker tag "$MIHOMO_IMAGE" "$LOCAL_MIHOMO_IMAGE"

printf "Please enter your Clash/Mihomo subscription URL: " > /dev/tty
IFS= read -r -s SUB_URL < /dev/tty
printf "\n" > /dev/tty

if [ -z "$SUB_URL" ]; then
  echo "Subscription URL is empty. Aborting."
  exit 1
fi

mkdir -p "$MIHOMO_DIR/proxy_providers"
chmod 700 "$MIHOMO_DIR"

TS="$(date +%Y%m%d-%H%M%S)"

cp -a "$COMPOSE_FILE" "$COMPOSE_FILE.bak.$TS"
cp -a "$SETTINGS_FILE" "$SETTINGS_FILE.bak.$TS"

echo "Backups created:"
echo "  $COMPOSE_FILE.bak.$TS"
echo "  $SETTINGS_FILE.bak.$TS"

MIHOMO_SECRET="$(openssl rand -hex 16)"

cat > "$MIHOMO_DIR/config.yaml" <<YAML
mixed-port: 7890
allow-lan: true
bind-address: '*'
mode: rule
log-level: info
ipv6: false

external-controller: 0.0.0.0:9090
secret: "$MIHOMO_SECRET"

profile:
  store-selected: true
  store-fake-ip: true

proxy-providers:
  sub:
    type: http
    url: "$SUB_URL"
    interval: 3600
    path: ./proxy_providers/sub.yaml
    health-check:
      enable: true
      url: https://www.gstatic.com/generate_204
      interval: 300
      timeout: 5000
      lazy: false
      expected-status: 204

proxy-groups:
  - name: AUTO
    type: url-test
    use:
      - sub
    url: https://www.gstatic.com/generate_204
    interval: 300
    timeout: 5000
    tolerance: 50
    lazy: false
    expected-status: 204

  - name: PROXY
    type: select
    proxies:
      - AUTO
      - DIRECT
    use:
      - sub

rules:
  - MATCH,PROXY
YAML

ch ```yaml
      - searxng-cache:/var/cache/searxng:rw
    depends_on:
      - valkey
      - mihomo
    networks:
      - searxng-net
    logging:
      driver: json-file
      options:
        max-size: "2m"
        max-file: "3"

  valkey:
    image: docker.io/valkey/valkey:8-alpine
    container_name: searxng-valkey
    restart: unless-stopped
    command: valkey-server --save 30 1 --loglevel warning
    volumes:
      - valkey-data:/data
    networks:
      - searxng-net
    logging:
      driver: json-file
      options:
        max-size: "2m"
        max-file: "3"

  mihomo:
    image: $LOCAL_MIHOMO_IMAGE
    container_name: searxng-mihomo
    restart: unless-stopped
    command: ["-d", "/root/.config/mihomo"]
    ports:
      - "127.0.0.1:7897:7890"
      - "127.0.0.1:9097:9090"
    volumes:
      - ./mihomo:/root/.config/mihomo
    networks:
      - searxng-net
    logging:
      driver: json-file
      options:
        max-size: "2m"
        max-file: "3"

networks:
  searxng-net:

volumes:
  searxng-cache:
  valkey-data:
YAML

cd "$APP_DIR"

if docker compose version >/dev/null 2>&1; then
  DC="docker compose"
elif command -v docker-compose >/dev/null 2>&1; then
  DC="docker-compose"
else
  echo "docker compose / docker-compose not detected"

## Hermes Only Keeps the Search Entry Point

Hermes doesn't need to know about Mihomo. It only needs to know that there is a SearXNG on the local machine:

```bash
nano ~/.hermes/.env

Write:

SEARXNG_URL=http://localhost:8888

Then specify the search backend in ~/.hermes/config.yaml:

web:
  search_backend: "searxng"

If you have the official Hermes SearXNG skill installed, you can continue using it:

hermes skills install official/research/searxng-search

If Hermes is user service:

systemctl --user restart hermes
systemctl --user status hermes --no-pager

There is also a boundary here: SearXNG is responsible for search, not for web page content extraction. The Hermes documentation also separates the search backend and extract backend. If you want Hermes to read the content of search result pages later, you still need to configure web.extract_backend with Firecrawl, Tavily, Exa, Parallel, or other extraction solutions.

Validation: Do Not Skip Layers

Test Mihomo first, don’t ask Hermes right away:

curl -I --proxy http://127.0.0.1:7897 https://www.gstatic.com/generate_204

A returned HTTP status indicates that the host’s debug proxy port is available. Then test the SearXNG JSON:

curl -s --max-time 50 \
  "http://127.0.0.1:8888/search?q=openai%20gpt&format=json" \
  | python3 -c 'import sys,json; d=json.load(sys.stdin); print(len(d.get("results", [])), "results")'

If a 403 appears here, first check whether search.formats includes json. If it times out or returns empty results here, then check the SearXNG logs:

cd /opt/searxng
docker compose logs -f searxng

Then send a request in Hermes that will trigger the search:

Please search online for the latest information on OpenAI GPT and list the source links.

If /search?...format=json appears in the logs, it means Hermes is already calling the local SearXNG. Next, let’s look at Mihomo:

cd /opt/searxng
docker compose logs -f mihomo

Focus on the subscription updates, provider, health check, and AUTO information. If Mihomo fails to even parse the subscription, then it doesn’t matter how correctly SearXNG is configured.

Boundaries You Should Not Omit

First, do not expose the port to the public network. The compose in this article only binds to the local machine:

ports:
  - "127.0.0.1:8888:8080"
  - "127.0.0.1:7897:7890"
  - "127.0.0.1:9097:9090"

Do not change it to:

0.0.0.0:8888:8080
0.0.0.0:7897:7890
0.0.0.0:9097:9090

Otherwise, the search service, proxy port, and Mihomo control port are all at risk of being abused on the public network. external-controller: 0.0.0.0:9090 is the listener inside the container, and what truly determines whether it can be accessed externally is the port binding in compose.

Second, if Google or Startpage consistently time out, do not rush to dismantle the entire chain. Keep Bing, Brave, DuckDuckGo, Wikipedia, and arXiv enabled for now; once the Mihomo subscription and health checks are stable, you can then re-enable the engines that are more prone to triggering CAPTCHAs or timeouts.

Third, the rollback points for scripts are straightforward:

cd /opt/searxng

cp docker-compose.yml.bak.your-timestamp docker-compose.yml
cp searxng/settings.yml.bak.your-timestamp searxng/settings.yml

docker compose up -d

I prefer this layered setup over wrapping the entire server in a global proxy. The problem with a global proxy is that its impact is too broad, and when something goes wrong, it becomes difficult to determine whether the issue lies in the system environment, Docker, application configuration, or the proxy node itself. Hermes, SearXNG, and Mihomo each only handle one thing, which actually makes troubleshooting easier.

References

写作附记

This piece retains the parts with the most operational value from the original material: the layered architecture, SearXNG JSON, explicit outgoing proxy, Mihomo provider and health checks, Docker Compose, Hermes configuration, verification commands, FAQ, and rollback. What was trimmed away are repeated explanations and absolutist statements that could mislead; for example, all://: is not a universally invalid configuration—it simply did not match as expected in that particular environment.

Additionally, one configuration name has been corrected: the Bing News engine for SearXNG should be written as name: bing news. If written as name: bing_news, it won’t match the default engine name, and the risk is higher than that of an ordinary spelling issue.

Original Prompt

The user provided a complete deployment material titled “Deploying Hermes + SearXNG + Mihomo on Servers in China: Routing Hermes Search Through Proxy and Automatically Selecting Available Nodes”, requesting to organize and analyze it, confirm the article content has no errors, and then invoke the blog writing skill to compose it into an article.

The material includes: background, target architecture, why not use system proxy, SearXNG settings.yml, Mihomo config.yaml, Docker Compose, transformation script, Hermes configuration, verification chain, frequently asked questions, rollback method, and final effect.

Loop engineering moves the person to the checkpoint

Sun, 14 Jun 2026 07:35:00 +0800

I attended MiniMax’s offline developer conference yesterday, and there’s one term that’s been stuck in my head ever since I got back: loop engineering.

At first, I thought it was just a matter of running the agent for a few more rounds. That understanding was too shallow. What really got me stuck is that AI writes code much faster than humans can review it. If human involvement remains stuck at “checking every step and making decisions at every step,” productivity will eventually be capped by the speed of human review.

This connects to my recent experience with Codex. When I was writing that goal article, I was more focused on the matter of “completion criteria”: what the goal is, where the boundaries lie, what to use for verification, and when something counts as done.

Looking back after the conference, “goal” is only a small but crucial part of loop engineering. It addresses how a single task can keep moving forward without constant human reminders. The bigger questions are: which tasks are worth handing over to a loop, which checkpoints must remain with humans, which judgments can be built into the system in advance, and which responsibilities cannot be pushed onto AI.

I now think there are two scenarios that fit best.

One is performance optimization. It naturally comes with metrics, and it’s best if it can also automatically retest. You tell the agent: the interface P95 needs to be reduced to a certain value, the first-screen rendering needs to be controlled within a certain number of milliseconds, the bundle size needs to be reduced to a certain number of KB, and regression tests must not fail. After each round of changes, it runs a benchmark, profile, or stress test; if the target isn’t met, it continues to look for bottlenecks. Here, the human’s value is not to check which line was changed each time, but to first provide verifiable acceptance metrics, and then judge at key checkpoints whether “this optimization has changed the business semantics.”

Another one is UI refactoring. The premise is having clear design specs, ideally achieving a 1:1 restoration. Without design specs, people keep getting pulled back into trivial judgments like “this margin is off,” “this color feels wrong,” “this interaction doesn’t feel right.” With design specs, AI’s loop can converge around screenshots, DOM, styles, and visual differences. People don’t need to chase every CSS adjustment, but they are responsible for the final interface.

The common ground between these two scenarios is that the deliverable is not just a vague “do it better.” Performance optimization has numbers, UI redesigns have design mockups. The loop works not because the AI is more obedient, but because the task itself has comparable goals.

Multica Is Like a Collaboration Board

At the conference venue, I also ran into people from the open-source community introducing their projects, among which Multica was quite interesting. When I looked it up afterward, I found that it isn’t a brand-new monolithic coding agent, but rather a layer that connects tools such as Claude Code, Codex, GitHub Copilot CLI, OpenCode, Gemini, Kimi, and Cursor Agent into a unified task collaboration layer.

In its README, agent is described as a teammate: it can be assigned issues, report blockers, update status, and also appear on the kanban board, in comments, and across the task lifecycle. More specifically in the docs: agent is a first-class member of the workspace, can be assigned issues, speak in comments, and be mentioned with @; when creating an agent, you can choose the underlying AI coding tool behind it, and configure instructions, model, environment variables, CLI arguments, visibility, and concurrency limits.

This turns “which model to use” from a temporary choice in a single prompt into a role configuration in team collaboration.

The most interesting aspect of it is not the phrase “multi-agent” itself, but rather the ability to assign different roles to different agents. For example, one agent can use a cheap model to handle repetitive fixes, another can use a more powerful model to make architectural judgments, and a third can focus solely on code review without making any changes; combined with mechanisms like Squad, a leader agent can route tasks to the appropriate members based on the content of the issue.

I haven’t run through Multica end to end, so it can only be treated here as a sample that can be cross-checked against publicly available material. But this design direction belongs to the same class of problem as loop engineering: rather than letting a single AI do everything, you put tasks, roles, models, review, and state transitions all into the same loop.

If a company is really going to implement AI to write code, this difference matters.

Now model prices vary, and so do their capability boundaries. Using the most expensive closed-source model for every task doesn’t necessarily keep costs under control; using cheap models for every task may push costs to the back end through rework and missed reviews. A more realistic approach is layering: low-risk, repetitive, clearly bounded work goes to cheap models first; critical changes, architectural trade-offs, and final review are reserved for stronger models or humans.

Using closed-source strong models for review is, I think, a good position. Review is not just “picking out grammatical errors” — it requires examining whether requirements have been corrupted, whether boundaries have been crossed, whether tests only cover the surface, and whether commit messages conceal risks. This type of task demands more judgment, but the call frequency may not be as high as during the implementation phase, making the cost accounting easier to accept.

Less Human Involvement Does Not Mean Less Human Accountability

There’s a point from the roundtable discussion that I really agree with: the code is written by AI, but the code is submitted by humans.

This sentence sounds like a reminder of responsibility, but it is also a principle of process design. AI can write code, modify code, run tests, generate PR descriptions, and even have another model review it first. But the person who ultimately merges the code into the main branch cannot say, “This was written by AI, it has nothing to do with me.”

What a company needs is not who is typing characters on a keyboard, but who takes responsibility for the result. Once you submit, it means you are willing to take responsibility for the business semantics, risk boundaries, test evidence, and rollback plan of this change. The deeper AI’s involvement, the less ambiguous this matter can be.

So loop engineering is not about completely removing people from the process. It’s more about rearranging where people fit in:

Before the task begins, humans clearly define the goals, boundaries, and acceptance criteria.
During the loop, the system and the agent handle repeated attempts, testing, fixing, and state synchronization on their own.
At critical checkpoints, humans review the evidence, the differences, and the risks, rather than chasing every line of the AI’s output.
When submitting code, humans take responsibility for the outcome and cannot use AI as an excuse to avoid accountability.

This is also why I connect seemingly scattered elements such as goals, performance metrics, design specs, and Multica together. They all advance, externalize, and structure human judgment. Humans participate a little less, but in more critical positions.

If this isn’t done well, it turns into another kind of inefficiency: AI produces a lot, humans can’t review it all, and the team ends up merging code based on “it feels about right.” When something goes wrong, the blame gets pushed onto AI. That’s not engineering—that’s just chaos with a different producer.

The loop that’s really worth pursuing is not about letting AI keep writing, but about making sure that every round of writing, modification, testing, and review can come back to the same acceptance checklist. Human work shifts from “watching it write” to “defining what counts as good, checking whether the evidence is sufficient, and taking responsibility for submissions.” That might be exactly where the term loop engineering has kept me thinking.

References

写作附记

Original Prompt

$blog-writer attended the Minimax offline developer conference yesterday. There’s something that has been on my mind—loop engineering. Recently I’ve been developing with codex and also enjoy using goals. I’ve written articles about goals before and discovered two scenarios where they fit best: performance optimization, where you directly provide the required performance metrics, and UI refactoring, where you provide design drafts and require 1:1 restoration. Going back to loop engineering mentioned at the beginning—programming in the AI era. Since AI productivity far exceeds that of humans, the speed of human review simply cannot keep up with the speed of AI output. However, with current vibe coding, many decisions still require human involvement. Once human participation increases, the entire process becomes difficult to fully automate. The concept of loop engineering is more about reducing the degree of human involvement.

I also met people from the open-source community who came to present their projects. One project was quite interesting—Multica. Check out the relevant materials and briefly introduce it. Apart from its design philosophy, the most interesting aspect is that different models are given different identities, which can effectively leverage various large models. After all, model prices vary nowadays, and using closed-source models for review is a good idea.

Regarding the implementation of AI code in companies, during the roundtable discussion, there was a consensus on one point: the code is written by AI, but the code is submitted by humans. You need to be responsible for the code you submit, which reflects your sense of responsibility and your attitude toward doing things. You can’t just say it was written by AI and has nothing to do with you.

Writing Approach Summary

Kept the in-person trigger of attending the MiniMax offline developer conference on 2026-06-13, without expanding it into a conference report.
Refocused the main thread on “how the role of human participation changes,” rather than explaining loop engineering, goals, Multica, and corporate responsibility item by item.
The Multica section only describes capabilities supported by public materials, and explicitly places “using closed-source models for review” within the author’s judgment.
Suppressed the elaboration of specific model pricing, company management systems, and code review processes, to prevent the article from drifting from workflow observations into management policy recommendations.

After Codex Fails at Writing UI, Let ChatGPT Draw the UI

Sun, 14 Jun 2026 05:14:22 +0800

At first I thought it was just that the frontend hadn’t had time to tidy things up. The business logic in strategy_studio has been moving forward: market data sync, PostgreSQL, asynchronous backtesting, report persistence, and the research workbench have all been wired in one after another, but the pages look more and more like “features got on board first, the UI buys a ticket later.” When I wanted Codex to do a destructive refactor of the UI directly, the real problem only then surfaced: no matter how harsh I made the prompts, the interface still looked like components being shifted around inside the old grid.

This wasn’t the first time I used Codex to modify code. For tasks like backend pipelines, API integration, test fixes, and report generation, I usually just throw the objective in, let it read files, modify code, run validation, and commit. The problem arose with this frontend refactor: I hadn’t done a UI draft first, and the business structure had already been laid out. Once the old interface exists, what the model sees is not just the requirements, but also ready-made components, ready-made routes, ready-made styles, and a bunch of historical inertia.

My initial reaction was also a typical one: just keep adding prompts. Allow destructive refactoring, redesign the UI and interaction logic, don’t make the elements too dense, adapt it for 24-inch and 32-inch monitors, focus on core features, and don’t be conservative. Every sentence looked correct on its own, but taken together they still lacked one thing: what it should ultimately look like.

What’s Stuck Isn’t the Button Color

strategy_studio is not a small tool with only two or three buttons. The README already positions it as a Chinese-first strategy research platform: a Next.js frontend, FastAPI API, PostgreSQL, Worker, Scheduler, and the Yahoo synchronization pipeline all running together. The frontend README further converges the pages into a single research workbench: sample preparation, experiment configuration, task tracking, result review, template maintenance, as well as report details and operations/maintenance entry points.

The front-end issues of this kind of project, on the surface, are “not good-looking enough,” but in reality, the information hierarchy hasn’t been established. Should users look at market data first, or look at backtesting tasks first? Among the curve, indicators, input parameters, and re-run button in the report details, which is more important? Is the homepage a navigation page or a workbench? Should the large screen be filled up, or should it leave some breathing room?

These instructions are described only in words, and Codex easily interprets them as “make some tweaks to the existing page.” It will modify components, adjust spacing, change colors, but still follow the old structure. Allowing destructive refactoring isn’t enough either, because destructive refactoring addresses whether the code can be heavily modified, not whether there is an acceptable visual goal.

I went out during the day, and suddenly thought of something in the evening: don’t let Codex keep guessing the UI. First, draw the UI design mockup on the ChatGPT web client, then hand the image over to Codex to recreate it.

The prompt used at the time was very straightforward:

https://github.com/ttf248/strategy_studio Analyze the front-end code of the current project, understand the business logic, and as a professional UI designer and interaction designer, help me redesign a set of front-end pages. Produce a separate design for each page, and make sure they are adapted for 24-inch and 32-inch monitors. The elements should not be too dense, so that users can easily focus on the core functions.

The result produced by this step is much more stable than continuing to pile up adjectives in Codex. It first provides a visible target: how the page is laid out, how the module proportions are divided, where the visual center is, and roughly how much whitespace is used. When Codex takes over afterward, the task shifts from “redesigning based on aesthetics” to “understanding the business from the diagram and recreating the code.”

The color scheme should also be turned into material first

The first round of diagrams solved the layout issue, but the default color scheme is still the standard blue-and-white of a regular admin system, which doesn’t quite match my aesthetic. If at this point I just throw “I don’t like blue-and-white, switch to a more sophisticated color scheme” to Codex, it’s essentially still letting it guess.

This is the initial blue-and-white color scheme. It already illustrates the page structure, but the overall feel is still rather generic and resembles an ordinary admin dashboard.

I changed my approach afterward: first, I discussed the color scheme direction on the ChatGPT web client, making the base color, accent color, card hierarchy, and the feel of a research tool clear, then asked the image model to show its intent based on this direction. It is not an official Figma design, nor is it a complete design system, but it is already sufficient to lock down the things most likely to drift during the front-end refactor.

What we ultimately got was not a single “big-and-comprehensive homepage” design, but design drafts broken down by page: Homepage, Market Research, Backtest Experiment, Report Center, Report Details, Strategy Templates, Operations & Maintenance. This breakdown is critical. The frontend of strategy_studio is not a marketing page but a research tool; the Homepage, Market page, and Report Details page each serve different reading tasks and cannot all be crammed into the same set of card grids.

Once the design comes in, Codex’s role becomes clear. It doesn’t act as a UI designer from scratch; instead, it reads the existing business logic, breaks down components, adjusts layouts, aligns interaction states, runs lint/build, and then refactors the legacy frontend to a target state that closely matches the design mockups.

Not just a “smarter web version”

Later, I checked the official materials again, trying to confirm whether this issue could be explained as “ChatGPT has different intelligence capabilities across different channels,” or as “Codex has its features gutted when used directly.” That explanation is too crude.

OpenAI’s description of Codex CLI is a coding agent in the local terminal: it can read code, modify code, and run commands in the current directory. Codex web, on the other hand, handles code tasks in a cloud environment. When OpenAI discusses the Codex agent loop, the focus is not on “the model coming up with everything out of thin air,” but rather on how model reasoning, context management, tool invocation, file read/write, and command execution together constitute a software task.

ChatGPT Images is another entry point. According to OpenAI Help’s description of Images in ChatGPT, users can create and edit images in conversations, and also create mockups or creative visuals. This product form is naturally more suitable for exploring visual goals first, because its deliverable is the image itself, rather than immediately diving into code.

So the difference this time isn’t “who is smarter,” but rather that the task forms are different. Hand “redesigning the UI” directly to Codex, and it will think from within the code repository: which files need to be changed, which components cannot be broken, and how to make the build pass. Hand the same business context first to ChatGPT Images, and it will first compress the abstract aesthetics and information hierarchy into a visible target image.

This also explains why Codex later became easier to use. It’s not that the design mockups bypassed Codex, but rather that the design mockups supplemented Codex with context and acceptance criteria. The official Codex prompting documentation also emphasizes that you need to provide Codex with relevant files, images, and clear completion standards. Without images, “the elements shouldn’t be too dense” is just an adjective; with images, it becomes a reproducible layout constraint.

Smoother Chain

After this, I’ll break down this kind of frontend refactoring into three parts.

In the first stage, start with visual exploration. Have the ChatGPT web client read the repository, understand the business, and generate diagrams per page, then iterate on color schemes, density, large-screen adaptation, and visual hierarchy. The output of this stage is not code, but target diagrams.

In the second step, hand over the target screenshots and engineering constraints to Codex. The prompt should not stop at “redesign the UI”; it needs to clearly specify which images are the targets, which business logic must not be broken, which pages are allowed to undergo breaking refactors, and which tests and builds must pass.

The third phase enters engineering restoration. This is where it becomes appropriate to have Codex run in goal or continuous task mode: first read the frontend structure, then split the implementation by page, run verification after each page is completed, and finally perform a unified check of interactions and responsiveness.

I used to treat “writing harsher prompts” as the solution. Looking back now, what’s more useful in frontend refactoring isn’t harshness, but filling in the missing intermediate artifacts. Without UI mockups, disruptive refactoring just breaks things apart further; with UI mockups, disruptive refactoring finally has direction.

This workflow is not only suitable for strategy_studio. Any personal project that already has business logic but whose frontend has long been patched together as it goes along can be handled the same way: first let the image model draw out the interface targets, then let Codex turn those targets into code. The models are not replacing one another; each one takes on the segment that better matches its own kind of work.

References

写作附记

Original Prompt

In daily use of codex for calling the ChatGPT model to write code during the development of the open-source project https://github.com/ttf248/strategy_studio, I started off without creating UI mockups and went straight into business logic development, which resulted in an unplanned front-end interface. I tried writing prompts directly in codex — allowing destructive refactoring, redesigning the UI and interaction logic — but no matter how I adjusted them, the results were never quite right. At that point, I still hadn’t suspected anything wrong with codex itself. During the day, I went out, and at night, inspiration struck: what if I used the web version to call image2 to generate a set of UI design mockups, then handed them off to codex running the goal model to faithfully reproduce them in code? I gave it a quick try and it worked without any issues. Here is the prompt I used:

https://github.com/ttf248/strategy_studio Analyze the current project’s front-end code, understand the business logic, and act as a professional UI designer and interaction designer to help me redesign a complete set of front-end pages. Generate a separate image for each page, making sure they adapt to 24-inch and 32-inch monitors. Keep elements from being too densely packed so users can focus on the core functionality.

At this point the results were already pretty good, but the default color scheme — the conventional blue-and-white — didn’t match my aesthetic taste. So I continued refining the approach: first, we discussed and agreed on a suitable color palette through web chat; then, image generated a demo based on that palette; finally, combining the color scheme with the previous prompt, we got this:

https://github.com/ttf248/strategy_studio/tree/main/ui

I’ve written a lot of content — please organize it and wrap it up into a single blog post. For any questions you’re unsure about, search the web on your own for reference. The repository at https://github.com/ttf248/strategy_studio/tree/main/ui contains a lot of images; feel free to pick a couple and embed them in the blog post.

Writing Approach Summary

This rewrite preserves the original’s judgment and materials, but changes the opening from “here is the complete answer first” to “let’s first step into the scene where the frontend refactor stalled.” The article trims away generalized model reviews and focuses only on the shift that actually worked in this workflow: turning the visual goal into a diagram first, then letting Codex handle the engineering restoration.

Codex goal embeds the completion criteria within the task itself

Wed, 27 May 2026 20:11:41 +0800

/goal is easily misinterpreted as a command to “let the agent work for a bit longer.”

This, of course, is merely its surface manifestation. If you give Codex a goal, it can continuously progress around that objective, instead of stopping after a single round of answers. But what is truly noteworthy is not how long it “runs,” but rather that it converts “what constitutes completion” from a temporary reminder into an intrinsic part of the task itself.

A standard prompt describes what needs to happen next. A goal, however, is more like attaching a checklist/acceptance form to an agent: What is the objective? Where are the boundaries? Which validations must pass? What conditions must be met for it to be considered complete?

goal is not the continue button

If we only look at its command form, /goal resembles an enhanced version of “continue until complete.” However, this shifts the focus.

The hardest part of long tasks isn’t whether the model wants to continue, but who decides whether or not to proceed after each round.

When you ask an agent to migrate a frontend project, it might stop simply after adjusting the routes, believing the job is done; when you ask it to fix a test, it might just make the current failing use case pass and quit; and when you ask it to rewrite a batch of articles, it might think it’s handled all the critical parts by only tweaking a few high-risk drafts.

These stopping points may not be incorrect, but they often differ from human acceptance criteria (or: human standards/judgments).

The goal is to solve this: clearly define the acceptance criteria beforehand, so that every subsequent round can judge against it.

A broad objective is as follows:

/goal Help me migrate the frontend to Next.js

Its problem isn’t that its output is too short, but rather that it lacks a definite termination/stopping condition. Codex can rewrite multiple pages, can restructure components on the fly, and also continuously fill in whatever content it deems necessary.

A more usable way would be something like this:

/goal Migrate the order management backend from React Router to Next.js App Router.
The visual behavior of the login page, order list, order details, and checkout page must be consistent with the old version.
Do not change the API contracts or database schemas.
After completing a batch of pages, run npm run build, npm test, and Playwright critical paths.
Only when all these validations pass can it be considered complete.

The extra details in this paragraph are not filler; they constitute four control planes:

Element	Role
Goal	What is the final desired outcome?
Boundary	Which interfaces, data, files, or behaviors cannot be handled manually?
Validation	What evidence proves that it was truly completed?
Termination	After fulfilling what conditions can it stop?

These four things are what raise the goal.

Why Does It Run For A Long Time?

Codex can continue to make progress on the goal, not because a single answer was elongated.

The actual workflow resembles a loop: plan, execute, observe tool results, revise, and then decide whether or not to continue. Build failures, test failures, screenshot inconsistencies, lint errors, or failed evaluation samples will all send the task back to the next iteration.

If the goal specifies a verification method, the agent cannot simply declare it “should be fine” based on intuition. It must obtain evidence. If the evidence hasn’t returned, further investigation continues; if the evidence fails, remediation is required; only when all evidence passes can the work be signed off.

This is why goal is suitable for tasks such as transfer, refactoring, batch revision, prompt eval, and troubleshooting long pipelines. Their common characteristic is that they cannot be completed in a single pass, and their completion cannot rely solely on subjective judgment.

Conversely, such a goal is very dangerous:

/goal Think of a more advanced product solution

It lacks boundaries, validation, or a stopping condition. The agent might run for a long time, but running for a long time does not equate to usefulness. At a minimum, you must clearly state how many sets of solutions are produced, what constraints are covered, what criteria are used for filtering, and when it should stop.

Claude Code is also handling the same thing

Claude Code also has /goal. The official documentation explains it more directly: The user sets a completion condition, and Claude will continue working across turns until the condition is met.

The Claude Code documentation also mentions that at the end of each round, it checks whether the completion condition is met; if not, it proceeds to the next round. This point is critical because it externalizes the action of “proceeding” from the model’s own subjective conclusion and turns it into an additional conditional judgment.

The specific implementation details of the two companies do not need to be forced to be equivalent, but their direction is consistent: the terminal agent is moving from “executing the next instruction” towards “continuously progressing around verifiable goals.”

We can simply categorize it as:

Capability	What problem it addresses	Suitable Scenarios
`/goal`	Defines explicit completion criteria and advances to verifiable results across multiple rounds	Migration, Refactoring, Batch Fixing, Long-running tasks
`/loop` or looping capabilities	Allows the same task to execute repeatedly based on count or condition	Retries, Generating candidates, Batch exploration
hooks	Automatically executes rules on fixed events	Formatting, Testing, Notification, Logging
Sub-agent/Multi-agent view	Decomposes tasks for observation and progression by different worker threads	Parallel analysis, Modular implementation, Long-term background tasks
Memory/Project README file	Solidifies long-term constraints and repository rules	Team standards, Code style, Tool entry points

In this table, the position of goal is very clear: it neither replaces hooks nor replaces memory. It manages “what level constitutes completion for this task.”

Goals should be written like acceptance criteria

I will now write a goal out in four lines:

Objective: What final user-visible result must appear.
Boundaries: Which files, interfaces, data, visual elements, or behaviors cannot be modified/tampered with.
Verification: What commands, tests, screenshots, evaluations, or manual checks serve as evidence.
Stopping Condition: Stop when all conditions are met; pause when encountering specific permissions, facts, or product judgments.

This is very different from a regular prompt.

A regular prompt is more like a next step action, while a goal is more like the completion criteria. It’s not about making the human disappear from the process; rather, it’s about embedding human judgment upfront. You no longer have to remind it round after round that “this isn’t complete,” but instead write down what constitutes completion as mandatory conditions from the very beginning.

Therefore, the more you want the agent to run autonomously, the narrower you must define its goals.

The less you focus on monitoring the process, the more rigorous the validation documentation must be.

The more you don’t want it to deviate, the clearer you must write the boundaries.

This is where goal truly deserves attention. It’s not about adding more commands, nor is it how long it can run; rather, it’s that the terminal agent is beginning to bring “who determines completion” into focus.

References

Author’s Notes

Original Prompt

$blog-writer Detail Codex's newly released goal command: what is its working principle, why does it run for such a long time, and what are the official examples? Does Claude Code have similar naming/commands? Additionally, compile the useful and popular new features recently released by two terminal tools into a table.

Writing Idea Outline

Keep the original main judgment: The core of goal is not the command name, but the completion condition.
Reinsert the Claude Code comparison and terminal function table required in the original prompt.
Eliminate “official document recitation” and focus on how to write a usable goal.

How do NVIDIA data center GPUs iterate after the release of ChatGPT?

Fri, 15 May 2026 19:58:51 +0800

Let’s first establish the date. ChatGPT’s public research preview version was released on November 30, 2022, not 2023. [1]

After this point, NVIDIA’s data center GPU main roadmap is quite clear: The conclusion of Ampere, followed by Hopper taking over. Hopper focused on expanding VRAM capacity and refresh rate, while Blackwell will shift its focus from “single-card dense compute power” toward “inference throughput, power consumption, and system-level interconnection.” The China-specific versions represent a different story: A800, H800, and H20 are fundamentally compliance versions created under US export control constraints, and therefore cannot be viewed using the same metrics as the global flagship line.

This only counts two lines:

Global Data Center Training/Inference Mainline: A100 as the control baseline, H100, H200, B200, B300.
China Dedicated Line: A800, H800, H20.

I didn’t include the L4, L40, L40S, and L2 in the main text either. It’s not that they aren’t important; it’s just that they are more related to the video inference, general inference, graphics, and virtualization line, and mixing them with the large model training main line of A100/H100/H200/B200 would confuse the pricing and performance metrics.

Look at the Main Storyline

To conclude: If we only look at the release cadence after November 30, 2022, H100 was the true starting point for the early generative AI explosion; H200 was a refresh card designed to “fill the memory gap”; B200 represents what truly constitutes a platform-level generational replacement, and B300 pushes Blackwell one step further into the inference and reasoning era.

Model	Release Date	Architecture	VRAM	VRAM Bandwidth	Interconnect	Official Performance Metrics
A100 80GB	Nov 2020, as comparative baseline	Ampere	80GB HBM2e	2.039 TB/s	NVLink 600 GB/s	BF16/FP16 Tensor Core 312 TFLOPS, INT8 624 TOPS [2]
H100 SXM	2022-03-22	Hopper	80GB HBM3	3.35 TB/s	NVLink 900 GB/s	BF16/FP16 1,979 TFLOPS, FP8 3,958 TFLOPS; DGX H100 single node 32 PFLOPS FP8, a 6x increase over DGX A100 [3][4]
H200 SXM	2023-11-13	Hopper Refresh	141GB HBM3e	4.8 TB/s	NVLink 900 GB/s	The focus provided by Nvidia is not doubled core compute power, but rather 1.9x inference for Llama2 70B and 1.6x for GPT-3 175B; relative to H100, it features larger and faster memory [5][6]
B200 SXM	202

What’s easy to misunderstand here is that the H200 is not a “brute-force double compute card”; rather, it is more like supplementary knowledge built upon the Hopper era. Once large model training and inference enter the stage of ultra-long contexts, massive KV caches, MoE (Mixture of Experts), and larger batch sizes, the bottleneck is no longer solely determined by the BF16 peak number, but rather by VRAM capacity and VRAM bandwidth. The H200 addresses this shortcoming.

The true generational leap happens with Blackwell. Blackwell is no longer just selling a single card; it is selling an entire suite of platform capabilities: new precision, interconnectivity, system-level bandwidth, inference cost, power efficiency, and rack-scale architecture. This is why when many sources discuss the B200, the single-card metrics are not as immediately understandable at a glance as those for the H100. This is because NVIDIA’s narrative focus has shifted from “how many TFLOPS does this card have” to “what size model can this entire system run and what will the cost be.”

Take Another Look at Chinese Specialty Lines

The China exclusive line must be viewed separately. Because its goal is not to defeat global flagship cards, but rather to stay below the export control red line while retaining commercial usability.

The most memorable sentence from this discussion is: A800 and H800 are more like “reducing interconnectivity,” while H20 involves “continued restrictions even on computing capability.”

Therefore, if someone only looks at the memory specification and draws the conclusion that “H20 is newer than H800, so it must be stronger,” this judgment is flawed. While H20’s 96GB HBM3 and 4.0 TB/s bandwidth look impressive, its very existence is predicated on meeting stricter export limitations. Its commercial goal is first to be sellable, and only secondly to maximize usability.

What are the upgrades compared to the previous generation?

First, let’s talk about computation methods:

\[ \text{Upgrade Rate}=\frac{\text{New Indicator}-\text{Previous Indicator}}{\text{Previous Indicator}} \]

However, this formula is only suitable for metrics with consistent definitions. VRAM, memory bandwidth, and NVLink bandwidth can be calculated directly; however, platform-level inference cost and whole machine throughput cannot be forced back into the single-card TFLOPS metric framework.

Global Key Themes

Generation	Biggest Change	Quantifiable Upgrade Magnitude
A100 80GB -> H100 SXM	Tensor Core and memory bandwidth rise together	Memory capacity 0%; memory bandwidth from 2.039 to 3.35 TB/s, approx +64.3%; NVLink from 600 to 900 GB/s, approx +50%; BF16/FP16 from 312 to 1,979 TFLOPS, approx +534.3% [2][3]
H100 SXM -> H200 SXM	Focus shifted to “larger and faster memory”	Memory from 80GB to 141GB, approx +76.3%; memory bandwidth from 3.35 to 4.8 TB/s, approx +43.3%; NVLink remains basically the same; BF16/FP8 peak metrics remain basically unchanged [3][6]
H200 SXM -> B200 SXM	Platform leap from Hopper to Blackwell	Memory from 141GB to 180GB, approx +27.7%; memory bandwidth from 4.8 to up to 8 TB/s, approx +66.7%; But the biggest change is FP4, 1.8 TB/s NVLink, and entire system and rack-scale inference efficiency [8][9]
B200 SXM -> B300 SXM	Blackwell Ultra pushes large memory and reasoning further forward	Memory from 180GB to 288GB, approx +60.0%; publicly reported memory bandwidth remains up to 8 TB/s; DGX B300 dense FP4 is 1.5 times better than DGX B200, with attention improving by 2x [10][11]

Upon reading through, a pattern emerges:

H100 is the generation that aggressively boosted single-card tensor computing power.
H200 is the generation focused on supplementing VRAM capacity.
B200 is the generation that transformed “training cards” into “AI factory infrastructure.”
B300 is the generation that more clearly pushes Blackwell towards reasoning and large-scale inference.

China Exclusive Line

Generational Gap	At first glance seems like an upgrade, but actually needs to be viewed separately	My judgment
A800 -> H800	If only looking at local HBM bandwidth, from the A100 generation to the H100 generation, this can roughly be understood as a +64% generational advancement.	But the core constraint is still interconnection, not single-card local memory.
H800 -> H20	Memory increased from 80GB to 96GB, approx. +20%; if based on common public parameters, bandwidth increased from 3.35 to 4.0 TB/s, approx. +19.4%.	This is not a pure upgrade. H20 is a compromise product due to greater compliance pressure and cannot simply be treated as an “H800 Plus.”

This also explains why the China special supply line is not suitable to frame as “how much every generation comprehensively improves.” This specific product line inherently carries compliance constraints; its design objective is not technical optimization, but commercial feasibility within regulatory constraints.

How Much Has the Price Actually Increased?

This section is most susceptible to inaccuracies/fabrication. Because NVIDIA rarely publicly discloses the single-card MSRP for data center GPUs, what is more commonly available in the public domain is:

DGX complete system price or third-party listed price for the entire system.
Channel quotes for China-special edition cards/GPUs.
Media, securities firm, or supply chain news.

Therefore, here I only provide “publicly traceable price samples,” rather than fabricating an official price list that looks complete but is actually inconsistent in its scope/criteria.

Object	Public Price Sample	Interpretation compared to previous generation
DGX H100	Official starting price of $199,000 at release on 2022-03-22 [4]	This is the cleanest official anchor point.
DGX H100	Market listing

So, concerning “how much the overall selling price increased,” I offer two conclusions:

First, the global flagship line has indeed risen significantly. Based on publicly comparable samples, the DGX B200 is roughly 40% to 50% more expensive compared to the DGX H100 listed during the same period. [19]

Second, China’s special allocation lines are not characterized by continuous price increases; rather, there might be a situation where “later released cards are cheaper.” The open quote price for the H20 eight-card server is approximately 30% lower than the H800 eight-card server. The reason is not ethical consideration (or conscience), but that the performance capability has been further compressed. [17]

Final Wrap-up

If I had to summarize the shifts in NVIDIA data center GPUs following the release of ChatGPT into a single sentence, my take is that:

The H100 was the catalyst for the generative AI boom; the H200 is a memory-oriented refresh, but the B200 is the true generational leap for the AI factory era, while the B300 clearly paves the way for the reasoning era. The China-specific lineup operates on an entirely different logic. It is not chasing flagships, but rather focusing on maintaining usability within regulatory gaps.

Do not view these two criteria together/conflate them. If you mix them, it is easy to draw conclusions such as “the new card has larger VRAM, therefore its generation is stronger,” or “the price is lower, therefore the cost-performance ratio is higher.” These are conclusions that seem superficially plausible but are based on incorrect premises.

References

Author’s Notes

Original Prompt

Compile a summary of NVIDIA GPU models and their corresponding performance parameters released since the launch of ChatGPT, detailing how much each generation upgraded from the previous one and the overall price increase. I specifically need data center GPUs, including special versions for China.

Summary of Writing Ideas

Fix the actual release date of ChatGPT to November 30, 2022, to avoid misalignment of timelines from the start.
Separate Nvidia data center GPUs into “Global Flagship Mainline” and “China Customized Line,” and do not force these two lines into a single generational upgrade history.
Percentage upgrades should only be calculated for directly comparable metrics, primarily VRAM, memory bandwidth, and interconnects.
For pricing, do not fabricate MSRP for individual cards; instead, only adopt official starting prices, complete system list prices, and Reuters channel quotes.
L4, L40S, and L2 are not expanded upon because they tend to mix the training mainline with the general inference/graphics line.

Brainstorming Expansion

Area	Inclusion Status in Main Text	Rationale
A100 as baseline	Included	The user asks for “previous generation vs. previous,” and without A100, it’s impossible to calculate the upgrade magnitude of H100.
L4, L40, L40S, L2	Rejected	Belongs to data center products, but is more oriented towards video/graphics and general inference, inconsistent with the main training price scope.
GB200, GB300 full system architecture	Partially Included	Used to explain why Blackwell starts emphasizing platform-level performance rather

AI can write code, what will newcomers use to level up?

Mon, 27 Apr 2026 23:18:36 +0800

In the last few months, while writing code using tools like Claude or Codex, my most striking realization wasn’t that “programmers are obsolete,” but rather that many tasks that used to be given to newcomers for practice can now generate a basic first draft themselves. Whether it’s creating a scaffold, adding several tests, or making small modifications on the fly—after running through these operations, the speed is genuinely fast, so fast it feels almost bittersweet.

For someone like me, who graduated ten years ago, frankly, this is more about increasing efficiency. Because I generally know where it’s reliable and where it isn’t; where something looks functional but actually has pitfalls hidden further down the line. But for fresh graduates, this topic isn’t so straightforward. AI isn’t just here to take over a few hours of manual labor; it feels more like it is compressing the traditional path of how a newcomer goes from zero knowledge to proficiency. This is also why I wanted to write about it separately.

What’s actually fading away isn’t the programmer, but mobile phone skill training

In the past, our team always had a batch of tasks that weren’t technically complex but were perfect for new hires. These included modifying pages, integrating interfaces, completing CRUD operations, fixing minor bugs, and debugging by following logs little by little. The work itself was small, but it was enough to take someone from merely “knowing how to write code” (syntax) to truly understanding “how the live system actually works.”

Currently, this type of work is the most susceptible to being taken over by AI.

The U.S. Bureau of Labor Statistics (BLS), in its updated Occupational Outlook for 2025, provided a very interesting comparison. On one hand, it states that roles such as software development and testing will continue to grow over the next decade; but on the other hand, it clearly indicates that the position of computer programmer—which is more focused on “writing and executing code”—is declining, as many repetitive programming tasks will continue to be automated. This change is critical.

It’s not that the software industry doesn’t need people; rather, the value derived solely from “taking tasks and writing code” is increasingly diminishing. Corporations, of course, are focused on cost efficiency: if AI can generate a first draft, followed by an experienced person to refine it, why do they still need to staff a slew of junior positions merely for slow mentorship, like in the past?

So, what might be truly frightening may not be layoffs themselves, but often the fact that companies are no longer replenishing staff or expanding roles. The door is still there, but the opening has become narrower.

Why Seasoned Professionals are Best Positioned to Capitalize on AI Dividends

It’s quite paradoxical. On the surface, code generated by AI might not be better than that written by fresh graduates. However, those who truly master using AI are often seasoned professionals who have learned through many pitfalls and failures.

The reason is also not complex.

First, seasoned professionals know that “looks runnable” is not the same as “truly deployable.” In Anthropic’s Economic Index from January 2026, when they analyzed software development in isolation, they found that although these requests are highly standardized, the task success rate was only about 61%. Furthermore, most scenarios still require back-and-forth iteration; it cannot simply be resolved by dumping the entire thing to AI once. METR conducted an even more sobering experiment in July 2025, allowing open-source maintainers who had been familiar with the repository for years to use cutting-edge AI tools of that time. The result was actually a 19% overall slowdown. This indicates that when the context is complex, quality standards are high, and boundaries are fuzzy, AI is not autonomous driving; it is more like a very talkative intern.

Secondly, experienced developers have something they call “code taste.” This concept might sound somewhat abstract, but it is very tangible. Whether an interface should be split this way, whether exceptions should be swallowed like this, whether testing is merely a trick to fool the CI, or if refactoring will preemptively bury pitfalls for the next week—these are things that matter. AI makes mistakes now, and many of those mistakes aren’t syntax errors; they are conceptual errors, abstraction errors, or boundary condition errors. Honestly, it is even harder for someone who hasn’t experienced traditional programming to identify these types of mistakes.

So, in the age of AI, what is most valuable is not typing speed, but critical thinking (or judgment).

Newcomers aren’t without paths, they are just out of the old ones.

I don’t believe that newcomers will be without opportunities. The World Economic Forum’s January 2025 report still ranks software and application developers among rapidly growing roles, and the U.S. Bureau of Labor Statistics’ long-term outlook for software development positions also shows growth. This suggests that the demand has not disappeared entirely.

But the entry method must have changed.

The previous default path was to start with some grunt or difficult work, learning as you went while coding, and relying on sheer time and effort to build your intuition. Now, companies are more likely to expect that upon joining, you already know two things.

One thing is to treat AI as a tool, not as an answer. You must be able to clearly articulate the problem, break down the requirements, and prompt it to give you usable drafts first.

Another thing is being able to review it effectively. It’s not enough to just say, “This is wrong”; rather, you need to know where the mistake lies, why it is wrong, and how to modify it so that it aligns with the project context.

This is troublesome. Because the ability to “review code” is something that usually takes years of working experience to develop, now that opportunities to practice are decreasing, yet newcomers are instead required to possess this skill much earlier. How should I put it? It’s like a game that dismantled the newbie village, but the boss is still waiting for you up ahead.

What Will It Be Like in the Future

My current judgment/assessment is relatively simple.

AI will continue to boost the output of experienced professionals while also continuing to diminish those parts of work in entry-level positions that are easiest to standardize and break down. These two things are likely to happen simultaneously, and they do not conflict. The ILO’s report from May 2025 was quite measured, suggesting that generative AI is more likely to change the structure of tasks rather than eliminating entire jobs all at once. The issue is that when the task structure changes, the first things to be eliminated are often those low-risk tasks originally designed for training new employees.

Therefore, the most scarce talent going forward is not merely someone who “codes best,” but rather someone who possesses foundational abilities, can harness AI, and takes ownership of both business outcomes and quality.

For seasoned professionals, this feels like adding a sharper shovel—it’s still taxing, but significantly more efficient. However, for beginners, the issue isn’t whether they should use AI; it’s that if they rely on it from the start, who will tell them when it writes well, and when it is producing convincingly plausible nonsense?

Neither the school nor AI can provide this answer right now. Ultimately, you’ll probably have to figure it out yourself. Ugh, the door isn’t closed, but getting through it is much harder.

References

Writing Notes

Original Prompts

Some thoughts on AI programming. As a recent graduate, my coding abilities are certainly not comparable to Claude or Codex. For experienced developers, AI can boost efficiency, but it has also led companies to significantly cut the positions of junior programmers. I graduated ten years ago and have gone through the process of starting as a beginner and reaching senior levels. What about those who enter the field now? How will they fare? The older generation lived through the days of “old-school” programming, so frankly speaking, they can recognize whether AI wrote good code or bad code, because current AIs still make mistakes.

Summary of Writing Ideas

Center the main argument on the idea that “AI is compressing the learning ground for newcomers, not shrinking the entire software industry.”
In the first half, first discuss why experienced workers benefit more from the AI dividends, and then explain how this can coexist with newcomer anxiety.
Use materials/data sets like BLS, WEF, Anthropic, METR, and ILO to firmly establish two points: “structural job changes” and “the necessity of human judgment in AI.”
Intentionally avoid elaborating on specific job search strategies or creating a ‘how newcomers should learn’ list of motivational advice.
The conclusion must return to the core reality: it is not that there are no opportunities, but rather that the traditional “newbie training ground” is disappearing.

After moving to Unicom, the US nodes aren't as good anymore.

Thu, 16 Apr 2026 01:56:14 +0800

I recently moved, and the broadband at my place switched from China Telecom to Unicom. Usually, when I watch dramas or play games, I don’t feel any noticeable difference. It wasn’t until a couple of days ago that I tried to download some materials and habitually switched to the US node, but no matter what, the speed wouldn’t go up. I was a bit dumbfounded at the time.

I figured this out later. I used to think that because the bandwidth in US data centers was sufficient, the US nodes were naturally stronger. Now I see that this understanding is only half right. Having ample server resources in the US is one thing; which broadband provider to use domestically, how the international exit path works, and whether the return trip benefits from better backbone connections are the other half. Issues that weren’t exposed when I used Telecom were all revealed after switching to Unicom.

It’s really hard to notice in daily life

These kinds of differences are easily hidden in normal circumstances.

Watching shows relies on the video platform’s own scheduling, and often cross-border bandwidth doesn’t need to be kept at full capacity. Gaming is more concerned with latency jitter and node stability; when it comes to downloading large files or running high bandwidth continuously, whether the line is suitable immediately reveals its flaws.

So now I actually feel that the download speed is the most honest benchmark. Just because you usually feel everything is normal doesn’t mean this connection is truly wide; it just means you haven’t pushed it to its limit.

Previously, US nodes could be fully utilized; it’s not just the US data centers that are powerful.

In the publicly available materials from China Telecom, ChinaNet is its main network, and CN2 (AS4809) is its next-generation global backbone network that emphasizes low latency and high quality, targeting businesses that place greater importance on international quality. Many people have heard of this; even airport vendors often use it as a selling point.

So, previously when I was on the telecom broadband, switching to a US node could boost my download speed. Looking back now, it’s probably not that “America is inherently faster,” but rather that the combination of “US data center resources + upstream lines at the node + China Telecom’s international routing” just happened to align perfectly. Once the line hits a direction favorable to China Telecom, the experience really is great.

To put it plainly, at that speed before, it wasn’t necessarily that the nodes were generally stronger; it was more like I happened to be standing on the side that had an advantage.

China Unicom doesn’t lack international internet, but the route might not be the same as your original one.

However, this matter cannot be simply understood as “China Unicom relies entirely on renting telecom networks.”

China Unicom also has a considerable international network. The official website of China Unicom clearly states that its backbone network, AS4837, has over 400 PoPs globally, and it also has AS9929 for higher quality capacity. In terms of coverage, it has laid out points in Singapore, Taiwan, and the United States.

But the problem is this. “Having international internet access” and “the proxy node you bought right now happens to be friendly to this carrier” are not the same thing.

What home broadband users truly feel is not just whether the carrier has overseas resources, but also:

Which carrier is local to you?
Which return path does the proxy service provider’s upstream prefer?
Are the outbound and return paths of the nodes optimized in the same way?
Is there congestion during peak hours?
Is this link more biased towards China Telecom, or more biased towards Unicom/China Mobile?

Many US nodes; in the past, I could saturate them when using Telecom, but it doesn’t work after switching to Unicom. I am more inclined to understand it as: the line combination of these nodes was inherently more friendly to Telecom. I didn’t feel this before because I myself was on Telecom.

Why Singapore and Taiwan are Doing Better

This is also quite interesting.

After switching to Unicom, the US nodes are no longer working well, but Singapore and Taiwan have improved quite a bit. This change actually illustrates the issue better than just saying “US node speed dropped.”

My understanding is that the Asia direction was originally closer, with shorter paths, and denser regional interconnection. Since Connect has network resources and PoPs in both Singapore and Taiwan, if the proxy service provider provides a more direct regional path at the Asian nodes, the download speed will naturally be easier to achieve.

This doesn’t necessarily mean that Singapore or Taiwan data centers are stronger than the US, nor does it mean that all broadband connections should choose an Asian node. It’s more like your current broadband has a better line match with these regional nodes.

How should I put it? Whether a node is strong or not isn’t just about its physical location in the data center; you also have to consider if the route home recognizes you.

This finally corrected my previous understanding

My previous thought was quite simple: US data centers are large, and broadband resources are abundant, so choosing a US node for downloading materials should basically never be wrong.

Looking at it now, this judgment is too coarse.

A more accurate way to say it is:

Whether the US node can be fully utilized depends not only on the US data center, but also on which backbone network your broadband provider is connected to.

The line that works very well for China Telecom users might not work the same way when switched to Unicom. For Singapore and Taiwan, which are actually smoother under Unicom, it’s not that “Asian nodes suddenly got stronger,” but rather that the lines finally fit better.

So in the future, when buying proxies or choosing nodes, I probably won’t automatically focus on the US anymore. It’s more important to first look at what kind of broadband you have and then see which line the service provider favors, rather than whether the data center is in the US or Asia.

Ah, this kind of difference is really hard to tell when just binge-watching dramas or playing games. It only becomes obvious when you have to transfer large files.

References

Writing Notes

Original Prompt

Prompt: In mainland China, when using proxy software for scientific internet access, I noticed a difference between Unicom and Telecom broadband. It’s hard to notice during daily activities like watching dramas or gaming. Occasionally downloading materials, I prefer selecting US nodes because the download speed there can basically reach full capacity. This aligns with my previous understanding that servers in US data centers are provided with ample broadband resources. Recently, after moving and switching to Unicom bandwidth, I discovered a problem: the download speed on US nodes cannot go up anymore. Switching to Singapore or Taiwan has improved things a lot. This is related to China’s backbone network; CN2 high-speed backbone networks are all operated by Telecom, while Unicom has to rent Telecom’s network.

Writing Outline Summary

Keep the real trigger point of “only realizing the difference after moving and changing broadband” and do not write the article as a general network science popularization piece.
Keep the core judgment: “It’s about how fast the node is, and you can’t just look at US data centers; you also need to consider domestic broadband and international routing.”
Supplemented verification on CN2, China Unicom international backbone, and overseas PoPs, correcting “China Unicom relies entirely on renting Telecom network” to a more accurate description.
For the part about Singapore and Taiwan being faster, explain it as “shorter regional paths and higher line matching degree,” rather than stating it as an absolute rule.
Structurally, first write the subjective feeling (experience), then explain the reasons, and finally bring the conclusion back to “how to choose nodes in the future.”

Changing the architecture, Hermes and OpenClaw tokens are not consumed in the same way.

Thu, 16 Apr 2026 01:13:53 +0800

After finishing the article “[Mistaking Hermes for an OpenClaw Alternative, Maybe I Was Biased From the Start]"(/en/p/hermes-openclaw-not-the-same-game/), I went through a round of documentation on both sides. The more I read, the more I felt that to truly see the difference between these two things, just looking at the features isn’t enough; looking at how tokens are consumed is actually more direct.

I’ll state my judgment first.

OpenClaw is by default more like a long-term online workbench; many identities, rules, workspace files, and message constraints naturally persist across conversation rounds, so the base model is usually heavier. Hermes, on the other hand, is noticeably more restrained; much of the context is discovered and injected on demand, and the system prompt deliberately maintains a stable prefix, making it easier to control token usage by default.

Of course, this doesn’t mean that Hermes is necessarily more cost-effective. If you enable the memory provider, skills, sub-agent, and long tool output all at once, it can burn through tokens just as fast. But frankly speaking, these two architectures have not been consuming tokens in the same way since day one.

Why is OpenClaw by default heavier?

OpenClaw’s design logic was never “to create a lightweight agent for a quick chat,” but rather “to build up a long-term, persistent agent workbench.” This is stated very clearly in the official workspace documentation.

AGENTS.md, SOUL.md, and USER.md are loaded for every session. These files, such as IDENTITY.md, TOOLS.md, HEARTBEAT.md, BOOT.md, and MEMORY.md, also revolve around the same workspace. The documentation even specifically reminds that HEARTBEAT.md should be kept very short to avoid token burn. This reminder itself indicates that OpenClaw is well aware that its default context is quite thick.

However, one thing that needs to be made clear is that OpenClaw doesn’t have everything permanently resident in the context. Its official token documentation clearly states that when skills are added to the system prompt, they are by default just metadata, and specific instructions must be explicitly read as needed. Therefore, its issue isn’t “not knowing how to save,” but rather that “the base it defaults to carrying is inherently larger.”

This is actually not difficult to understand. OpenClaw aims to solve the issues of “long-term online presence” and “multi-message surface visibility.”

You have to make an agent live simultaneously in places like Telegram, Discord, Slack, and WhatsApp. It also needs identity, routing, boundaries, and delegation. Many rules cannot just be tacked on temporarily. It first needs to know who it is, how to speak, whom it’s facing, what the constraints of the current workspace are, where to find skills, which memories to carry, and which memories not to carry.

So, the token consumption for OpenClaw is more like a large fixed overhead.

Every time you send a message, you are not just saying one sentence to a model; you are activating an entire set of pre-configured “assistant environment.” This environment is very useful, but the cost is that the base prompt is thicker, and much context, even if not directly used in this round, will be placed there first.

Why Hermes Looks More Restrained

Regarding Hermes, the most interesting part of the documentation is that it keeps emphasizing two things: on-demand loading and prompt cache preservation.

First, look at the context files. Hermes only loads the project contexts that match the current working directory when a session starts; things like AGENTS.md, CLAUDE.md, and .cursorrules are first-match wins, so they won’t all be loaded at once. More importantly, the AGENTS.md in a subdirectory isn’t read entirely at startup; it is discovered and injected incrementally only when you actually navigate to that directory, read that file, or reach that path.

The official documentation clearly writes out the benefits of this design:

no system prompt bloat
prompt cache preservation

This taste is very different from OpenClaw. Hermes seems to be saying that it’s best not to make the system prompt too long; place context that can be deferred, and don’t fill up the first round with things that only appear at relevant times.

The skills also follow the same logic. The Hermes skills documentation explicitly mentions progressive disclosure, with the goal of minimizing token usage. In other words, the skill is not permanently present in the full text by default; instead, the model first sees a lightweight index and only expands to show the detailed content when it’s truly necessary.

Furthermore, its memory system is also combinatorial. The official documentation severely limits the built-in memory: MEMORY.md is about 800 tokens, and USER.md is about 500 tokens, totaling a fixed capacity of around 1300 tokens. SOUL.md is for fixed identity, while USER.md and memory are included in the system prompt, but items like session search, memory provider, and Honcho feel more like external layers. This structure doesn’t naturally shorten the prompt, but it provides a default posture that is easier to control cost-wise.

So, from an architectural perspective, I would be more inclined to categorize Hermes as “default restraint, gradually increasing weight.”

It’s not about who is more advanced, but where the cost is placed

Many comparison articles like to write token consumption as a single conclusion, which I think is incorrect.

OpenClaw is heavier, but that doesn’t mean it’s poorly designed. On the contrary, it intentionally front-loads a lot of features. If you want an assistant that is long-term online, cross-platform, has personality, has a workspace, and has routing, then these contexts will eventually cost something. It just chooses to bring all these things together on the default path first.

Hermes is more restrained, but that doesn’t mean it has no overhead. Once you stack a long SOUL.md, the project AGENTS.md, multiple skills, MCP, memory provider, sub-agent, and long tool outputs together, tokens can still roll very quickly. Especially if the tool output itself is very long, or if you make it search back and forth in a complex codebase, that’s not something you can save just by naming the architecture.

So, a more accurate way to say it is:

OpenClaw moves the cost to “assistant environment residency.”

Hermes shifts the cost to “capability on demand.”

Neither approach is absolutely superior, but their billing structures are completely different.

What truly makes the difference is not just the system prompt

There are a few details that are quite easy to overlook.

First, Hermes was designed to stabilize the prefix in the context files. The subdirectory hints are appended to the results of relevant tools, rather than continuously feeding all project contexts into the system prompt. This approach not only saves tokens but also essentially leaves space for the provider’s prompt cache.

Second, tools like Hermes’s execute_code are designed to only return the final result printed by the script back to the model; all the intermediate results from RPC tool calls will not be included in the context. In complex workflows, this structure can eliminate a large amount of meaningless tool noise.

Third, OpenClaw’s workspace philosophy means it’s more like “bringing everything with you.” Even if many individual files are short, as long as there are many types of files, many layers of persona, many layers of rules, and many levels of skills and memories, the foundational context will gradually become thicker. This thickness is not a bug; it is part of its product design trade-offs.

Fourth, how both sides handle the sub-agent will also affect the total ledger. Many of Hermes’ designs emphasize local context and local execution, while OpenClaw emphasizes the organizational capabilities of multiple agent identities, routes, and delegates. One is more like switching context, and the other is more like organizing existence relationships. Finally, when it comes to the token billing, the shape is naturally different.

If you want to calculate it, don’t rely on your mouth

If you are really planning to use it long-term, don’t just listen to anyone saying “this is more energy efficient”; look directly at the data provided by the tool.

OpenClaw has commands like /context detail and /usage tokens, and the documentation will also remind you which files are injected at the start of each session. This is very suitable for seeing how thick the base knowledge really is.

Regarding Hermes, the official documentation provides clues such as token budget, skills progressive disclosure, and context file injection methods in Honcho and related features. You can look at this by combining hermes insights, session storage, and actual provider billing.

To put it plainly, the architecture can only determine “how to spend,” but it cannot decide “how much to spend.” What truly determines the bill is how long your SOUL.md is, how much memory you’ve stuffed in, how many skills you’ve enabled, whether the tool output was contained, and how expensive the model itself is.

I’m still on this side

If I only look at the default architecture and not extreme configurations, I still tend to think that Hermes is better at keeping token consumption within a relatively restrained range.

It’s not because it’s stronger, but because from the start of prompt design, it has been avoiding “meaningless persistent context bloat.” OpenClaw is the opposite; it would rather carry a bit more to ensure that the identity, workspace, and message boundary of that long-term online assistant don’t fall apart.

So if you are particularly concerned about token costs, and the workflow mainly happens in local, CLI, code repository, or skill knowledge base scenarios, Hermes’ approach will generally be more convenient.

If you want a long-term assistant that is truly available across various messaging platforms, then spending more tokens with OpenClaw feels like a normal cost. You can’t expect it to be online long-term like a person while also requiring it to be as lightweight as a one-off tool every turn.

Ah, this matter ultimately comes back to the old question.

Are you raising an online assistant, or are you raising a local agent core? Figure that out, and the token accounting won’t be so hard to understand.

References

Writing Notes

Original Prompt

Prompt: Hermes and OpenClaw write another article about their token consumption. Since the architectures are different, the consumption must be different.

Writing Idea Summary

Continue the judgment from the previous article, but narrow the focus to token consumption, and no longer repeat the overall architecture comparison.
Focus on comparing the default context assembly methods of both sides, rather than just talking about “who is more efficient.”
Hermes focuses on progressive disclosure, on-demand discovery, and prompt cache preservation.
OpenClaw focuses on workspace persistent files, long-term online assistant environments, and fixed overhead.
The conclusion retains the core judgment provided by the user, but adds a layer of practical reminder: to truly see the bill, one must combine official usage/context tools with actual provider billing.

Treating Hermes as a replacement for OpenClaw might be biased from the start.

Wed, 15 Apr 2026 23:52:20 +0800

Over the past couple of days, I’ve been flipping back and forth through the documentation for Hermes and OpenClaw, and the more I read, the more I feel that many people compare these two projects as if they are on the same level; in reality, the comparison is biased from the start.

They are all doing “personal AI assistants.” They can receive messages, call models, run tools, and retain some context. Hermes even specifically created hermes claw migrate, clearly knowing it would receive a batch of OpenClaw users.

But frankly speaking, Hermes is not a skin-deep version of OpenClaw, and OpenClaw is not just an agent framework with a few extra message entry points. One grows outward from the Gateway, while the other grows outward from the AIAgent. If you don’t grasp this difference first, discussing architecture, design philosophy, and ecosystem later will just lead to more confusion.

They were never intended to become one thing

The OpenClaw official documentation places Gateway in a very prominent position. Its definition is quite direct: a long-running Gateway holds all message planes, and the control plane clients, Web UI, and Node devices all connect to this Gateway via WebSocket. Furthermore, only one Gateway on a host manages the core message connections; sessions, like those used by WhatsApp, are explicitly monopolized by it.

What does this mean? It means that OpenClaw’s worldview is not “first have an agent, then connect several entry points to it,” but rather “first have a persistent communication and control plane, and then organize the agent runtime, session, skills, delegate, and sub-agent within this plane.”

Hermes is not like this. Its architecture diagram clearly shows these entry points: CLI, Gateway, ACP, Batch Runner, API Server, and Python Library, all converging into the same AIAgent core. Session storage uses SQLite + FTS5, and the tool backend, provider resolution, and prompt builder are all organized around this core.

So, in terms of architecture, OpenClaw is more like a “resident personal assistant system”; Hermes is more like an “agent runtime platform,” and the messaging entry point is just one extension surface of it.

One that treats the Gateway as the protagonist, and one that treats the Agent as the protagonist

This architectural difference will ultimately translate into a difference in user experience.

OpenClaw’s strength is in making the concept of “how to have a long-term, identifiable, bounded, and routable assistant across many channels” very robust. It has default DM pairing, security policies, routing by channel/account/peer, multi-agent binding, delegate mode, and even organizational-level issues like “who sends on behalf of, which account receives, and where it appears in the group” are specifically documented.

In other words, what OpenClaw wants to solve first is the existence problem. This agent must first act like a real person, or like a real organizational assistant, living on platforms such as Telegram, Discord, Slack, WhatsApp, Signal, and WebChat. Only then can you talk about whether it will write code or perform task distribution.

Hermes’ focus is more on “capability accumulation.” While it certainly has Gateway and can connect to platforms like Telegram, Discord, Slack, and WhatsApp, the most prominent terms in its official narrative are not routing or accounts, but persistent memory, skills, MCP, sub-agents, toolsets, as well as batch processing, trajectory export, and RL training.

This is very crucial. The core problem Hermes needs to solve is not “how to connect an AI to as many messaging platforms as possible,” but rather “how to let this agent gradually develop its own skills, memory, and reusable capabilities, and even be able to conveniently use them for training data and experiments.”

So I would prefer to summarize it like this:

OpenClaw is a communication-first personal assistant system.

Hermes is an agent-core-first self-hosted agent platform.

Memory, Skills, and Ways of Extension, It’s Not One Flavor

Many people will say, don’t both support skill, memory, and plugin? That’s right, they both do. But the focus is different.

In OpenClaw’s agent runtime, the workspace is a very heavy concept. Files like AGENTS.md, SOUL.md, TOOLS.md, BOOTSTRAP.md, IDENTITY.md, and USER.md are directly injected into the context in the first round of a new session. Skill loading also has an entire set of priorities: those in the workspace, those in the project, those in the personal directory, those in ~/.openclaw/skills, and bundled ones can all stack up. Essentially, it is building a unified long-term assistant environment of “personality + rules + workspace + channel.”

Hermes’s skills system is clearly more geared towards “procedural memory.” The official documentation states that skills are on-demand knowledge documents, following progressive disclosure, and are compatible with the open standard of agentskills.io, all unified under ~/.hermes/skills/. Furthermore, the agent itself can modify and delete skills. This feels less like “giving the assistant a few instruction manuals” and more like “crystallizing past actions into reusable capability blocks.”

Memory is the same. Hermes has built-in, very restrained MEMORY.md and USER.md, plus a session search using SQLite + FTS5, and it also supports 8 types of external memory providers. This design is quite interesting; instead of writing “long-term personality environment” in great detail right away, it separates short-term memory, retrieval-based recall, and external long-term memory.

OpenClaw is not without memory, nor is it without advanced capabilities like sub-agents, multi-agent systems, or delegation. On the contrary, it has plenty of these. However, based on its documentation structure and default mental model, it feels more like managing a continuously online assistant workspace; Hermes feels more like managing a continually evolving agent capability graph.

Why is the ecosystem drifting further and further apart?

In the ecosystem, on the surface, everything looks like “open source + extensible,” but in reality, they are quite clearly branched out.

OpenClaw’s ecosystem is broader and leans more towards the external access layer. Official plugins can expand channels, model providers, tools, skills, speech, web fetch, web search, memory, while community plugins and ClawHub are also growing around “connecting more platforms, connecting more accounts, connecting more workflows.” It even explicitly supports bundle formats like .codex-plugin, .claude-plugin, and .cursor-plugin. This move is very smart; essentially, it means absorbing the shells of other ecosystems first, thereby enlarging its entry points.

Looking at the OpenClaw documentation, you’ll find that it writes very extensively about sub-agents, delegates, organizational agents, thread binding, DM pairing, and group mentions. This indicates its ecosystem goal is not just for single-machine tinkering, but rather to become a truly long-running agent operating system.

Hermes’s ecosystem is more like “externalizing core capabilities.” It has a Python plugin system, a memory provider plugin, a context engine plugin, and MCP server integration, as well as open skill standards like Skills Hub and agentskills.io. Coupled with training-related interfaces such as batch runner, trajectory export, and Atropos, Hermes’s ecosystem naturally attracts two types of people.

One type is those who use it as a personal agent.

Another category is those who use it as a research, training, experimental, or workflow substrate.

The needs of these two types of users are very different, but the Hermes architecture can cover both. So you might feel that while Hermes’ ecosystem may not spread as widely in terms of “number of channels” and “message presence” as OpenClaw, I am more inclined to feel that its potential in the agent substrate area will be greater.

There’s also a pretty interesting detail. The official Hermes README doesn’t just have hermes claw migrate; it also features a community project, HermesClaw, which is used to run both the Hermes Agent and OpenClaw on the same WeChat account simultaneously. This signal is quite clear: Hermes isn’t pretending that it has no connection with OpenClaw; it is acknowledging the existence of this user migration path and then productizing this path as well.

My Final Judgment

If what you are looking for is a “personal assistant system that truly lives on the messaging front,” one that requires accounts, channels, routing, delegation, organizational boundaries, long-term online presence, and strong control planes, then OpenClaw’s path is more complete and feels more like a mature communication foundation.

If you are looking for an “agent core that becomes more like your own workbench the more you use it,” and what you value is accumulated skills, memory composition, MCP, plugins, sub-agents, training data, and subsequent moldability, then Hermes is more convenient.

So, stop simply asking “Can Hermes replace OpenClaw?”. That question is a bit rough.

A more accurate way to ask is:

Do you want an AI assistant that lives within a messaging network, or an AI platform that lives within local workflows and agent runtimes?

If you think this through, the choice won’t be so agonizing.

References

Writing Notes

Original Prompt

Prompts: The differences between Hermes and OpenClaw, architectural differences, differences in design philosophy, and ecosystem differences.

Writing Idea Summary

Remove the common question “Alternative Comparison” and directly set the main thread to “What is the core focus of the two projects?”.
For the architecture section, prioritize looking at the main entry points and core components in the official documentation, rather than secondary community summaries.
In the design philosophy section, avoid vague value judgments and focus directly on specific designs such as Gateway, Agent Core, memory, skill, and delegate.
For the ecosystem section, focus on comparing extension directions, not star ratings, popularity, or community hype wars.
Fact verification must be based on official repositories and documentation visible as of 2026-04-15.

AI writes demos quickly, and revisions are also really fast.

Fri, 10 Apr 2026 15:39:51 +0800

Recently, I started a small C++ project using AI. The most frustrating moment isn’t when it can’t write the code, but when it spits out a seemingly decent directory structure in three minutes and casually throws in a few third-party libraries, making the demo actually runnable. That’s the problem. You haven’t even figured out what the newly introduced library supports, how the compilation link works, or where its boundaries are, so rework is basically inevitable later on.

I’m increasingly feeling that what AI programming fears most is not the model being dumb, but starting too greedily. Especially with languages like C++, which don’t have much scaffolding to fall back on; if you miss one step upfront, later you have to account for several steps regarding compilation, linking, library versions, and directory structure.

My own conclusion on this is quite direct: AI is better suited as a pair programmer that can accelerate progress, rather than one that takes over all the initial design and dependency decisions for you. GitHub’s documentation suggests starting with simpler tasks, letting the agent master areas with clearer boundaries like bugs, documentation, tests, or technical debt; Anthropic’s Plan Mode documentation feels more like insurance for complex changes—first read the code, then create a plan, and only then decide whether to implement the change. The two approaches aren’t exactly the same, but they point in the same direction: don’t throw the hardest, messiest, most dependency-laden chunk at the AI right away.

So, this article won’t talk about a bunch of specifications; instead, it will provide a small case study that you can practice with immediately. You can follow along using any empty C++ repository.

It’s not that AI is impossible, it’s that the starting point was too greedy

Writing AI projects, especially in C++, is often a very rudimentary approach.

Start with a minimal working version from main.cpp
First, get the compilation pipeline running
Then decide which libraries to include
Ensure it still compiles at every step
Wait until the business outline is clear before starting refactoring

This method looks slow, but it’s actually very stable. Because at every step you know exactly what you gained, and you also know which step the problem originated from.

AI is prone to disrupting this rhythm. It really likes to “complete it for you” in one go:

I’ve also set up the configuration files for you.
I’ve also extracted the logging module for you.
I’ve also organized the error codes, utility classes, and directory structure together.
It even selected the integration method for third-party libraries for you.

The result is that the demo looks complete, but your mental model is empty. If there’s any discrepancy between a library’s capability boundary and what you expect later on, rework won’t be in one spot, but a series of them.

Start with a small project to get some practice

I think it would be good for practice to build a very small log scanning tool. You can name it anything, like logscan.

It does one thing: it scans log files in a directory, counts the number of error and warn, and then prints the result. This problem is not big, but it’s just enough for you to go through the most common parts of a small C++ project:

main and parameter entry point
Output formatting
Logging
Configuration files
Business module decomposition

The key is not how advanced the questions are, but how you break them down.

I will break it down into four steps, and for each step, the AI is only allowed to do the current small part.

Step One, Prepare the Output First

First, don’t touch the logs, don’t touch the configuration, and don’t think about abstraction.

Do two things:

Create a minimal main.cpp
Include fmt, and print the scanned directory and statistics results

The official documentation for fmt provides very clear CMake usage. It has two targets: fmt::fmt and fmt::fmt-header-only. The documentation also explicitly recommends the compiled version, and the reason is quite simple: it’s more friendly to build times. CMake’s FetchContent documentation is also very suitable for starting new projects because it fetches dependencies during the configure stage, rather than downloading them temporarily during the build stage.

In this step, I will directly fix the dependency strategy and not let the AI play freely.

Either unify FetchContent
Or unify find_package
Don’t use vcpkg one moment, add_subdirectory the next, and manually copy headers after that

If it’s a practice project, I usually constrain the AI like this:

Do not optimize the architecture all at once.
The current goal is only this step, and the project must always be compilable.

This step involves only 3 things:
1. Include fmt in the root CMakeLists.txt.
2. Use fmt in main.cpp to print the passed directory and scanning results.
3. Keep the existing target names and directory structure; do not add new logging, configuration, or testing frameworks.

First, give me a modification checklist, then provide the code.
After completion, provide the compilation command and expected output.

There’s a detail here that is quite important. If AI is not constrained, it can easily treat “doing something extra in the back” as a bonus point. You must explicitly tell it that it cannot go beyond the scope this round.

Step Two, Upload Logs

Once the first step is done, it can be compiled, run, and the output looks clean, then add spdlog.

The official spdlog README is actually very straightforward about what it can do. It can handle console logging, as well as regular files, rolling files, daily rotation, and even a backtrace ring buffer. The problem is this: with so many capabilities, AI gets easily excited and wants to cram everything into the first version—colored console output, rolling logs, daily archiving, global logger factory, all at once.

No need.

For this step, I will only have it do console logging, or at most add the simplest file logging. Because what you really need to learn at this point is not “how to design a complete logging system,” but three things:

Where to initialize the logger
How to get the logger in business logic
How to separate logging and normal output when an error occurs

If you immediately implement rotating_file_sink or daily_file_sink, while you quickly gain a “feature-rich” logging module, you might not actually know if you need rotation or daily splitting, or if printing to the terminal during debugging is sufficient.

To put it plainly, understanding how to use spdlog::info() is more important than making the logging factory look pretty.

Step Three: Touching the Configuration File

The configuration file section is easier for AI to turn into a large project.

I prefer to practice with a simpler library like toml++, not because it’s the “most powerful,” but because it’s very suitable for the initial stages of a new project. It is itself a C++17 header-only TOML parser, and the examples in the README are also quite short; you can directly read with toml::parse_file("configuration.toml"). More importantly, its single-header mode is incredibly convenient. The official description is quite interesting: the single-header approach is just “throwing toml.hpp into your source tree,” and then there’s “no second step.”

For this step, do not use a “Configuration Center,” but read three fields:

Scan Directory
Keyword List
Write Output to File

Then put them into a very thin AppConfig struct.

That’s enough.

A lot of rework, it actually comes down to this. You haven’t even figured out if the configuration should be read once at startup or hot-updated at runtime; whether it’s only for the local CLI or if you plan to reuse it for service processes later. AI has already taken five or six layers ahead of you. If you change direction later, all that “complete design” from before will just become a burden.

Step Four: Decomposing Business Modules

For libraries like fmt, spdlog, and toml++, please read through the official documentation yourself, run them successfully in practice, and then ask AI to help you break down the modules.

When you break it down like this, then you’ll have a clear understanding.

I usually receive the following types of files:

main.cpp is only responsible for assembly
app_config.{h,cpp} is only responsible for configuration reading
logger.{h,cpp} is only responsible for log initialization
log_scanner.{h,cpp} handles business logic

Note that the splitting happens at this step, not from the first step. In the first step, just lining up the table of contents neatly is often only visually pleasing and doesn’t necessarily mean it’s cognitively clearer.

What’s truly useful isn’t some complex specification

Many online AI coding guidelines are very complicated, with dozens of rules. It’s tiring just looking at them. The ones I kept for myself are actually only three, and they are enough.

First, create a “resource card.”

Every time I prepare to reference a third-party library, I first ask the AI to answer these four questions:

What problem does this library solve?
How far has the official support reached?
Which 1 or 2 capabilities am I planning to use this time?
What new constraints will it bring to building, deploying, and running?

If you can’t explain this card, don’t play it yet.

Second, only let the AI pass one checkpoint at a time.

For example, in this round, it is only allowed to complete:

Can code
Can run
Can see expected output

Don’t add things like “while I’m at it, let’s refactor the directory,” “I’ll quickly add a configuration class,” or “let’s unify the exception handling system” in the same round. In C++ projects, the more “conveniently” you do something, the easier it is to muddy the source of the problem.

Article Three, a rollback point must be left after every round.

Even if you don’t commit, you should be able to answer:

What exactly changed in this step?
If it’s wrong, which parts need to be deleted?
After reverting to the previous step, can the project still be compiled?

You will find that this line of thinking is actually very similar to manual development. The difference is that before you wrote the code yourself, now you are supervising the AI so it doesn’t run wild.

A More Practical Use Case for AI

If you really want to master this set of things, I suggest doing three consecutive rounds.

In the first round, only let the AI help you with fmt, and read through the CMake configuration and output code yourself.

Round two, only let the AI handle spdlog. You decide whether it should be console logging or file logging; you cannot use both together.

Round three, only let the AI handle toml++, and you decide the configuration items yourself; do not allow the AI to guess future requirements for you.

After going through these three rounds, your feeling about AI programming will change a lot. Before, it was “how did it generate so much for me all at once,” but later it will slowly become, “how small should I prompt the next step to be most worthwhile.”

I think this is the most valuable skill in AI programming right now.

It’s not about creating a demo.

but rather to steadily move a project forward.

To be honest, this set of methods isn’t very novel; it’s even a bit old-fashioned. But old methods are often resistant to rework. Especially with a language like C++, if you let the AI do less and confirm more yourself at the beginning, things will genuinely become much easier later on. Otherwise, you might fly up in three minutes only to have to go back to the drawing board for three hours—and the one who looks foolish is still yourself.

References

Writing Notes

Original Prompt

Prompt: Writing a project manually, especially for languages like C++ that don’t have scaffolds. First, create a basic framework starting from main and gradually expand the logic by adding modules, such as configuration files, logging modules, and iterative business modules. My habit is to ensure it compiles at every step and refactor incrementally; what I write initially might not be the optimal solution. Regarding AI programming, often nowadays, steps are skipped, and too little thought is put into the early stages, especially when introducing new third-party libraries. While AI can quickly generate a demo, because I don’t fully understand the details of newly introduced third-party libraries or what functions they support, the rework rate for related logic is significantly higher than manual development. Do you have any good practical examples that I can learn from? I know there are many complex AI programming guidelines online, but I want something simpler and quickly implementable.

Writing Idea Summary

Narrow the topic down to the tooling lane, focusing on where things become easier or conversely more awkward after AI changes workflows.
Don’t write a general checklist of best practices; instead, turn it into an incremental hands-on case study using a small C++ project.
Keep the core judgment that the user “starts from main, can compile at every step, and refactors gradually,” and make this the main narrative thread throughout the article.
Supplemented with primary sources for Anthropic, GitHub, CMake, fmt, spdlog, and toml++ to avoid guessing about tool capabilities and dependency methods.
The article structure progresses according to fmt -> spdlog -> toml++ -> module decomposition, intentionally preventing AI from achieving everything in one go.

VS Code for C++, don't forget CMake and GDB Printer

Thu, 09 Apr 2026 19:21:36 +0800

Previously, when I debugged C++ in VS Code, the configuration basically stopped at launch.json, maybe with an extra line for GDB. Fill in the program, fill in the gdb, set the breakpoints. And then what? Then every time before debugging, I had to manually run cmake --build in the terminal. What was even more annoying was that after setting breakpoints on custom prices, contracts, or order types, the VS Code debug window often only showed a bunch of internal fields. The data was correct, but it wasn’t human-readable. The ridiculous part is that some tutorials I saw before stopped around launch.json. It wasn’t until recently, when I had an AI set up a new project for me, that it conveniently added preLaunchTask and gdb_printers.py, that I realized: debugging C++ in VS Code isn’t just about starting GDB. You can automatically trigger CMake compilation before debugging, and after hitting a breakpoint, you can even let GDB load a Python script to format business types into something readable for us. Honestly, this isn’t some black magic. But it perfectly filled two annoying gaps in daily C++ debugging: pre-launch build and variable display after a breakpoint.

Only Half Was Configured Before

The .vscode/launch.json in many tutorials is probably something like this:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug with GDB",
      "type": "cppdbg",
      "request": "launch",
      "program": "${workspaceFolder}/build/debug/bin/vscode_cpp_debug_demo",
      "args": [],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "MIMode": "gdb",
      "miDebuggerPath": "/usr/bin/gdb",
      "setupCommands": [
        {
          "description": "Enable pretty-printing for gdb",
          "text": "-enable-pretty-printing",
          "ignoreFailures": true
        }
      ]
    }
  ]
}

This part cannot be wrong. The official C++ example in VS Code also uses a similar structure: type uses cppdbg, MIMode uses gdb, and pretty-printing is enabled in setupCommands. But one crucial step is missing. When you press F5, it launches the executable file pointed to by program. The problem is, was this file just compiled? If not, then you are debugging the old program from last time. I used to do this stupid thing often. I changed the code and set breakpoints, debugged for a while, only to realize later: Oh, I forgot to recompile.

preLaunchTask is the hook

VS Code Tasks are essentially about hooking external commands into the editor. Compiling, testing, and packaging—what used to be typed in the terminal can now be written into .vscode/tasks.json. Then, preLaunchTask in launch.json finds that task by its label. For a minimal CMake project, it could look like this:

.
├── .vscode/
│   ├── launch.json
│   └── tasks.json
├── CMakeLists.txt
├── src/
│   └── main.cpp
└── tools/
    └── gdb_printers.py

Here is the CMakeLists.txt first:

cmake_minimum_required(VERSION 3.16)

project(vscode_cpp_debug_demo LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

add_executable(vscode_cpp_debug_demo
    src/main.cpp
)

And here is .vscode/tasks.json:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "cmake: configure debug",
      "type": "process",
      "command": "cmake",
      "args": [
        "-S", "${workspaceFolder}",
        "-B", "${workspaceFolder}/build/debug",
        "-DCMAKE_BUILD_TYPE=Debug"
      ],
      "problemMatcher": []
    },
    {
      "label": "cmake: build debug",
      "type": "process",
      "command": "cmake",
      "args": [
        "--build", "${workspaceFolder}/build/debug",
        "--parallel"
      ],
      "dependsOn": ["cmake: configure debug"],
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "problemMatcher": ["$gcc"]
    }
  ]
}

Notice the cmake --build here. The official CMake build entry point is this format: cmake --build <dir>. It calls the underlying native build tool, such as Make, Ninja, or MSBuild. This means VS Code doesn’t need to know whether your project should use make or ninja. CMake handles that part.

Before F5, first compile

Then go back to .vscode/launch.json and add preLaunchTask:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "CMake Debug with GDB",
      "type": "cppdbg",
      "request": "launch",
      "program": "${workspaceFolder}/build/debug/bin/vscode_cpp_debug_demo",
      "args": [],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "environment": [],
      "externalConsole": false,
      "MIMode": "gdb",
      "miDebuggerPath": "/usr/bin/gdb",
      "setupCommands": [
        {
          "description": "Enable pretty-printing for gdb",
          "text": "-enable-pretty-printing",
          "ignoreFailures": true
        }
      ],
      "preLaunchTask": "cmake: build debug"
    }
  ]
}

This line is the key:

"preLaunchTask": "cmake: build debug"

The flow when pressing F5 becomes:

launch.json
  -> preLaunchTask
    -> tasks.json: cmake: build debug
      -> dependsOn: cmake: configure debug
        -> cmake -S ... -B ...
      -> cmake --build ...
  -> gdb starts the latest program

If you are on Windows + MinGW, the program probably needs to be changed to a path with .exe, and miDebuggerPath should also point to your own gdb.exe. If it’s a multi-configuration generator like Visual Studio Generator, you usually need to add --config Debug to the build arguments. But the main flow remains the same. Before debugging, let VS Code run the build task first; don’t rely on memory alone.

Another Pain Point: Incomprehensible Custom Types

Just connecting the compilation link is not enough. If you wrap a business type slightly in your C++ code, the variable window on the left side of VS Code starts looking ugly. Take this example. src/main.cpp:

#include <cstdint>
#include <cstdio>
#include <string_view>
#include <vector>

namespace market {

struct Price {
    std::int64_t raw{};

    static Price from_double(double value) {
        return Price{static_cast<std::int64_t>(value * 10000)};
    }

    double to_double() const {
        return static_cast<double>(raw) / 10000.0;
    }
};

struct Instrument {
    char symbol[16]{};
    Price last;
};

Instrument make_instrument(std::string_view symbol, double price) {
    Instrument instrument;
    std::snprintf(instrument.symbol, sizeof(instrument.symbol), "%s", symbol.data());
    instrument.last = Price::from_double(price);
    return instrument;
}

} // namespace market

int main() {
    std::vector<market::Instrument> watchlist{
        market::make_instrument("IF2406", 3578.6),
        market::make_instrument("IH2406", 2468.2),
    };

    market::Price limit = market::Price::from_double(3600.5);

    // Set a breakpoint here to observe watchlist and limit.
    return watchlist.empty() || limit.raw == 0;
}

In the business logic, market::Price represents a price. But what the debugger might default to showing is only:

limit = {raw = 36005000}

Yes, the data is correct. But in my head, I still have to manually convert 36005000 back to 3600.5000. If a project has a bunch of Price, Quantity, OrderId, Instrument, AlgoState, the debug window quickly turns into a structure graveyard. This is when you need GDB pretty-printers. The official GDB manual states it very clearly: It provides a mechanism that can use Python code to pretty-print values, and this mechanism works for both MI and command line. VS Code’s C++ debugging uses exactly this GDB/MI path.

Writing a Python Presentation Layer for Business Types

Create tools/gdb_printers.py:

import gdb
import gdb.printing


class PricePrinter:
    def __init__(self, val):
        self.val = val

    def to_string(self):
        raw = int(self.val["raw"])
        return f"{raw / 10000.0:.4f}"


class InstrumentPrinter:
    def __init__(self, val):
        self.val = val

    def to_string(self):
        symbol = self.val["symbol"].string()
        price_raw = int(self.val["last"]["raw"])
        price = price_raw / 10000.0
        return f"{symbol} last={price:.4f}"


def build_pretty_printer():
    printer = gdb.printing.RegexpCollectionPrettyPrinter("market")
    printer.add_printer("Price", "^market::Price$", PricePrinter)
    printer.add_printer("Instrument", "^market::Instrument$", InstrumentPrinter)
    return printer


gdb.printing.register_pretty_printer(
    gdb.current_objfile(),
    build_pretty_printer(),
    replace=True,
)

Then source it in launch.json’s setupCommands:

"setupCommands": [
  {
    "description": "Enable pretty-printing for gdb",
    "text": "-enable-pretty-printing",
    "ignoreFailures": true
  },
  {
    "description": "Load project pretty-printers",
    "text": "-interpreter-exec console \"source ${workspaceFolder}/tools/gdb_printers.py\"",
    "ignoreFailures": false
  }
]

Press F5 again. When you look at the debug window this time, the display for Price should be close to:

limit = 3600.5000

And Instrument will change from a blob of arrays and structs to something closer to the business logic:

IF2406 last=3578.6000

I deliberately say “close to” here because the pretty-printer needs to match the compiler, type names, and actual field layout. If your business types contain nested std::vector, std::array, or smart pointers, you might need to write more code in Python to access those fields. Don’t expect one script to cover everything. But the idea is solid. Separate what “you want to see when debugging” into its own layer of Python presentation logic.

Don’t treat launch.json as everything

Looking at this set of configurations now, I think there are at least three layers. .vscode/launch.json is responsible for “how to start the debugger.” .vscode/tasks.json is responsible for “building the project first before starting the debugger.” tools/gdb_printers.py is responsible for “displaying variables in a human-readable way after a breakpoint.” Before, I only configured the first layer. It worked, but it was very rudimentary. Smooth C++ debugging shouldn’t be like: “I have to compile it in the terminal first, then come back and press F5, and then manually calculate the meaning of fields in the variables window.” The more comfortable workflow should be: Press F5, VS Code automatically runs CMake, GDB automatically loads the project’s Python printer, and when a breakpoint is hit, the debug window directly speaks business language. What was most valuable about AI helping me set up this project wasn’t how much C++ code it generated. It was that it bridged these old tools together. It’s not that I didn’t know CMake before, or that I didn’t know GDB. I just missed configuring those intermediate hooks.

References

Writing Notes

Original Prompt

Prompt: $blog-writer vscode for C++ development, previous tutorials only showed simple configuration of gdb debugging information in launch.json, and didn’t mention configuring preLaunchTask. Being able to automatically trigger cmake compilation is a great feature when setting up new projects with AI; if the code has many custom data types that cannot be directly displayed in the VS Code debug window, one can introduce a Python script for preprocessing so that they can all be displayed. Regarding the above content, actual code examples are needed for everything.

Writing Idea Summary

The main thread should focus on the idea that “previously, only launch.json was configured, but it missed the two layers of glue: pre-build and runtime display during debugging.”
Keep the trigger point provided by the user: Discovering that when setting up a new AI project, preLaunchTask can automatically trigger CMake compilation.
Organize code examples around a minimal CMake project, including CMakeLists.txt, tasks.json, launch.json, custom C++ types, and GDB Python pretty-printer.
For factual verification, prioritize official documentation from VS Code, CMake, and GDB; the main text should not translate entire documents but only retain facts relevant to this debugging pipeline.
Provide a warning regarding the boundaries of the pretty-printer: Type names, standard library implementations, and field layouts can affect the Python script, so users must adapt it for their specific types in the project.

What is this process of layering folders and then wrapping them in a namespace called?

Thu, 09 Apr 2026 02:31:07 +0800

When I recently wrote the algorithm service, as soon as I implemented modules like twap and vwap, this old problem popped up again.

If we rely on class names to enforce semantics in C++, the naming convention quickly becomes out of control. Things like TwapOrderManager, VwapOrderManager, and AlgoOrderManager sound like, “I know my structure isn’t contained, but I’ll at least add a prefix.” Frankly speaking, organizing by folders and then adding a layer of namespace isn’t about code snobbery; it’s filling the gap that C++ has because it lacks Java’s native package system.

To state the conclusion first

If I have to find a software engineering term that is the closest, I think the two words that should be mentioned are:

Modularization
Namespace management

If I want to be more precise, this type of directory organization is often doing:

Packaging by feature / Packaging by domain, which is commonly referred to as package by feature
If the business boundaries are already very strong, it will also carry a bit of bounded context.

In other words, it is not a small concept with a single standard translation name; rather, it is more like several design principles stacked together.

What is actually solved here is not just about “looking good”

When many people first see a namespace, their immediate thought is to avoid name collisions. This is certainly true; the namespace in C++ standards was originally designed to prevent naming conflicts in large projects cppreference. However, when it comes to actual business code, its value goes far beyond that.

Once the directory structure and namespace are aligned, class names no longer need to repeat the higher-level semantics.

For example, previously one might write something like this:

class TwapOrderManager;
class TwapScheduleEngine;
class VwapOrderManager;
class VwapScheduleEngine;

After adopting directories and namespaces, a more natural way to write it is usually:

namespace algo::twap {
class OrderManager;
class ScheduleEngine;
}

namespace algo::vwap {
class OrderManager;
class ScheduleEngine;
}

This removes the redundant prefix, making the semantics clearer. Because “which context it belongs to” is no longer stuffed into the class name, but rather expressed by the directory and namespace.

Therefore, this approach essentially involves extracting contextual information from the names and letting the structure carry it.

This is more like “packaging by feature”

If your directory looks like this:

strategy/
  twap/
    order_manager.h
    schedule_engine.h
    slicer.h
  vwap/
    order_manager.h
    schedule_engine.h
    slicer.h

Then you are no longer doing traditional “packaging by technical layer.”

The traditional way is more like this:

models/
services/
utils/
controllers/

This structure looks neat at the beginning, but as the business logic grows, the service directory will look like a junk heap. Classes related to one feature get scattered across different directories, making it hard to read the code because your mind keeps jumping around.

Whereas the twap or vwap approach is closer to package by feature. This means a directory first answers the question, “What is this business module about?”, and then places all the objects and processes needed for that specific business module inside it. This concept is more common in the Java world, but it works just as well in C++; the difference is that what carries it isn’t a native package, but rather a combination of “directory + namespace + header file boundaries.”

Simply put, you are organizing code by the reason for change, not by the appearance of the code.

Taking it one step further, it’s about high cohesion and low coupling

The frequently mentioned phrase in software engineering, “high cohesion, low coupling,” actually falls into this area.

Since the objects under twap collaborate frequently with each other, they should be placed closer together, and their namespaces should also be close; vwap is similar to it but not quite the same, so it deserves its own boundary. Doing this has several direct benefits:

Class names within the same module can be shorter
Dependencies between modules are easier to see
It’s easier to judge during refactoring whether changes might propagate to other areas
If you eventually need to split a certain capability into an independent library, the cost will be smaller.

This is also why many mature projects end up looking like they have consistent “directory boundaries + namespace boundaries + compilation boundaries.” In projects like QuickFIX, you can see a similar idea: types and extension points are kept under clear namespaces rather than being artificially separated by very long class prefixes QuickFIX GitHub.

When It’s Not Enough Anymore

However, don’t mythologize this either.

Simply layering folders and wrapping them in a namespace can only indicate that you are moving towards modularization; it doesn’t mean the architecture is automatically sound. Several common pitfalls are also obvious:

Directory and Namespace Mismatch

The directory is twap/, but the code contains scattered global classes, or namespace common floating everywhere. This is basically useless.

Module Boundaries are False Boundaries

There are twap and vwap on the surface, but they actually #include each other. The common logic can leak through freely, and in the end, it just moves the files around without reducing any coupling.

The longer the `common` directory, the fatter it gets

This is the most common issue. Many projects are structured quite well at first, but later find it troublesome and start dumping things into common, base, or util. By the end, instead of having truly stable abstractions that have been properly developed, they end up with a massive pitfall that everyone can touch and everyone depends on.

Structure by Layer Only, Not by Business Domain

If your directory structure is always api, service, dao, model, then many business concepts actually have no home. Class names are forced to get longer and longer, cramming everything that should be expressed by the structure back into the name itself.

So, what should it be called?

If I were communicating this within a team, I think there are three levels of description:

If you just want to speak plain language: Modularization based on directories and namespaces
If you want to sound more engineering-focused: Package organization by feature, combined with hierarchical namespaces
If these boundaries are already strongly bound to business semantics: Module division with a bounded context feel

I personally lean towards the second option. Because it is closest to the scenario you described.

What you’re doing isn’t about C++ syntax tricks, nor is it just about naming conventions; you are doing something more substantial: Using structure to carry semantics, allowing names to return to their true meaning.

When class names get shorter and file names get shorter, what you first see when reading the code is an object with clear context like twap::OrderManager, rather than something that mixes directory responsibility, naming responsibility, and implementation responsibility all together like TwapOrderManagerImpl.

Only then does the code start to feel somewhat like Java after its package structure has matured.

The Last Sentence

So, in software engineering, there are corresponding concepts, but there isn’t one single standard answer.

Broadly speaking, it’s called modularization. In terms of code organization, it’s called feature-based packaging, plus namespace management. In terms of domain boundaries, it has something to do with bounded context.

How should I put it? The most valuable aspect of this practice is never that “the names look neat,” but rather that you finally don’t have to rely on super long prefixes to compensate for structural deficiencies.

References

Writing Notes

Original Prompt

Prompt: When developing code in C++, a good practice is to organize it by folders and then wrap each class with a namespace, similar to Java’s package. This can effectively reduce redundant content in file names and class names. A classic project is QuickFIX, and we have recently used a similar design for algorithm services. Many functional modules like TWAP and VWAP are similar. Is there a specific concept in software engineering for this?

Writing Approach Summary

Start by focusing on the actual trigger points of twap and vwap in algorithm services, making the judgment clear first.
Instead of forcing the problem into a single noun, it is broken down into more accurate levels such as modularization, namespace management, and packaging by feature.
The correspondence between QuickFIX and Java packages has been retained to illustrate that C++ often needs directories and namespaces to supplement structural semantics.
The focus is on “using structure to carry semantics,” rather than remaining at a superficial explanation like “avoiding name collisions.”
Added reference links for namespace, package by feature, and bounded context to facilitate further development.

Google has released Gemma 4 this time (III)

Wed, 08 Apr 2026 23:56:20 +0800

While browsing the forum this time, what struck me most wasn’t which company released another leaderboard, but a very basic statement: “Not enough VRAM; no matter how large the parameters are, it’s useless.”

Previously, I always understood “slow model” as a computational power issue. However, the more I read, the clearer it became that often, the problem isn’t that the GPU can’t compute it, but rather that the data cannot reside in the right place. Just by changing the memory path, the token speed doesn’t just slow down; it drops drastically.

The previous two posts covered the preliminary issues. First Post discussed the release and protocol, and Second Post explained why we should first look at 26B A4B on the 3060 12GB. This final post will only discuss how speed actually collapses.

Out of VRAM, and Not Just a Little Bit Slow

When you break down the process of inference, there are two particularly critical components:

Model weights
KV cache

The weights define the model itself, while the KV cache records the state of previous tokens. The longer the context, the larger the KV cache. As long as both these parts can be stably kept in the GPU VRAM, generating a token basically involves reading data, performing calculations, and writing back results within high-bandwidth memory, and the speed is usually quite good.

The real problem is when you run out of VRAM. Once it doesn’t fit, the inference framework has to compromise:

Storing some weights in system memory
Or storing some KV cache in system memory
Or even shuttling data back and forth between the CPU and GPU

At this point, the issue is no longer about “a little bit of extra computation,” but rather “waiting for data with every single token.”

Why the Cliff, Not a Linear Slowdown

When many people first encounter this pitfall, their intuition is that if the model goes from 14B to 31B, it will just be more than twice as slow, which they can tolerate.

Reality is not like that.

The real dividing line isn’t doubling the parameters; it’s whether or not the working set crosses the VRAM boundary.

As long as it hasn’t crossed, increasing model size usually results in a predictable slowdown.

Once it crosses, the system state changes:

Previously, it was an “on-chip closed loop” within VRAM.
Now, it becomes “VRAM + System RAM + Bus Transfer.”

Changing the path completely changes the cost. This is especially true during the decoding stage, which inherently proceeds token by token with a small batch size. At this point, what is most feared is having every single token wait for a batch of data to be transferred across devices.

So, you see a very typical phenomenon:

The model isn’t unable to run.
The GPU isn’t completely idle either.
But the tokens/s rate is extremely poor.

This is what people call the “cliff.” It’s not that the model suddenly got dumber; it’s that the memory path suddenly became inefficient.

Why `26B A4B` is Friendlier to Local Players

This also explains why I have consistently favored 26B A4B in my previous article. Its total parameter count is certainly not small, but only about 3.8B are actually activated per token. This means that under similar deployment conditions, the pressure it puts on computation and VRAM paths is often easier to control compared to dense large models. This isn’t magic. If your context window gets too long, or if quantization and framework support are inadequate, it will struggle just like any other model. However, compared to a dense model that immediately maxes out the entire VRAM, 26B A4B feels more like a realistic path for consumer-grade GPUs. So, often the issue isn’t whether 31B is weak, but rather which model is better suited for long-term coexistence with local hardware.

Why Macs seem “less prone to running out of memory”

What’s most different about Mac is not the model, but the memory architecture. Apple silicon uses unified memory. The CPU and GPU share a single pool of memory, unlike dedicated graphics machines where there are separate VRAM and main memory that have to be moved across a bus. The biggest advantage of this structure is that many models which would “not fit in VRAM at all” on dedicated graphics machines can take on a different state on Mac:

It might not be fast,
But it can probably fit into the memory first. In other words, Macs are less likely to hit that very rigid “VRAM wall” right from the start. This is why many people feel that Macs are particularly suitable for running large models locally as a fallback option. It solves the problem of “whether the entire working set can be loaded into the same memory pool.”

But Unified Memory Doesn’t Mean High Speed for You

You must look at this part separately. What did unified memory solve?

It solved the hard separation between dedicated VRAM and main memory.
It solved many cases where models couldn’t fit directly onto cards with small VRAM.
It solved some very ugly cross-device data transfers. But what didn’t it solve?
It didn’t change large model inference from “massive memory reads” to something else.
It didn’t make large models suddenly stop being bandwidth-hungry.
It didn’t automatically give all inference frameworks the mature ecosystem of CUDA. So, the comfort of Mac is not the same thing as the speed of a high-VRAM NVIDIA card. Mac is more like:
Quiet machine operation
Large total memory
Unified architecture
Finally allowing models that previously wouldn’t fit to run first. NVIDIA’s high-VRAM card is more like:
Mature ecosystem
Complete CUDA toolchain
When you truly keep the model and cache on the GPU, speed can be much easier to boost.

Why Speed Ultimately Depends on NVIDIA’s Large VRAM

This is not an emotional judgment, but a practical conclusion drawn after wrestling with local deployment. If you are pursuing these things:

Local assistant always running
Multi-turn long conversations
Long context windows
Higher token/s rate
Minimizing waiting time as much as possible Then ultimately, the large VRAM NVIDIA card is what matters. Because what you are truly buying is the ability to keep the model and cache stably on the GPU. Mac also works, but it is better suited for a different set of requirements:
I want a machine that can load an even larger model into it first.
I accept average speed, but I don’t want to mess with drivers and peripherals.
I care more about overall experience, power consumption, and noise. Both paths are reasonable; they just solve different problems.

Back to Gemma 4, My Final Verdict

This time with Gemma 4, I really feel that local open-source models have reached a stage where hardware discussion is more warranted. But just because the model gets stronger doesn’t mean the laws of physics loosen up accordingly. No matter how strong the 31B model is, it will slow down if VRAM is insufficient. Even with the practical 26B A4B, long contexts still put pressure on the system. And while Apple’s unified memory is comfortable, it only makes it easier to “get something running”; it won’t give you the speed of a large VRAM CUDA card for free. So, I’ll end with this rather blunt summary:

For speed, prioritize NVIDIA with large VRAM.
For reliability/fallback, Mac’s unified memory is indeed comfortable.
If you plan to run on hardware like the 3060 12GB long-term, don’t always aim for dense, massive models; a path like 26B A4B is more realistic. With this set of articles, I will wrap up here.

References

Writing Notes

Original Prompt

$blog-writer Google has released the Gemma4 model after a year. As usual, I'm trying to deploy it locally on that old desktop with an unupgraded NVIDIA 3060 12GB graphics card. This time I caught the initial release, but I couldn't find an upgraded version of the commonly used Gemma3. However, there is a similar version called GemmaE4b. Please first search and introduce all the models released this time, what the letters in their abbreviations mean, and then search for online reviews about Gemma4. The key point is that Google updated the model's protocol this time, so the restrictions for users are fewer. The biggest surprise: my usual test question—write a piece of C++ code to output a five-pointed star in the console. Last year's smaller parameter open-source models couldn't handle this problem, but Google managed it this time. In the first version, it gave the answer, completely exceeding my expectations. It knew about my trap; outputting a five-pointed star to the console is very difficult, so it directly hardcoded a string of a five-pointed star for direct console output. This is the original text: Because drawing a five-pointed star with precise geometric structure using mathematical logic in a pure text console (Console) is very complex (involving coordinate system transformation and pixel filling), the most classic and visually best method is to use ASCII Art. After I forced it to perform calculations, it also managed it through mathematical calculation, successfully drawing the five-pointed star. Previously, I often used Gemma4 for local translation tasks; many multilingual versions of historical articles on current blogs are like this. The model used for local testing: gemma-4-26b-a4b. The 31b version is indeed too slow. But looking at the reviews, the 31b effect is very good, and its ranking performance is excellent. While browsing forums, I realized that if the VRAM is insufficient and the model parameters are increased, the token generation speed will drop drastically. Can you explain why? Macs don't have this problem because they use unified memory; please explain the technical reason. Also, if speed is required, then an NVIDIA card with large VRAM is still necessary. The Mac solution can serve as a fallback, but it cannot match the speed. This content is very extensive; please evaluate whether it should be split into a series of articles.

Writing Outline Summary

The third article will only retain the two threads: “Why speed collapses” and “Why Mac does not equal fast,” without revisiting the content of the previous two articles.
Start by discussing VRAM limitations, then move to non-linear slowdowns; this logic is smoother than the previous version.
Mac and Nvidia will be discussed as two different dimensions—one focusing on reliability/fallback, and one focusing on speed.
The conclusion will only retain hardware judgment, without repeating the explanation of why series separation occurred.

Google released Gemma 4 this time (Part II)

Wed, 08 Apr 2026 23:52:20 +0800

If you only look at the leaderboard, 31B is definitely the most eye-catching. But when you actually get the machine out, it’s still that un-upgraded RTX 3060 12GB, and your judgment will change immediately. How should I put it? For local deployment, in the end, it’s not about who looks the fanciest, but who seems like the one you can live with long-term. For me, what is truly worth running first this time isn’t 31B, but 26B A4B.

The previous article Google released Gemma 4 (Part 1): Don’t rush to local deployment; you need to understand the model and protocol first covered the release and protocols. This current article only talks about the local experience itself; the last one continues with Google released Gemma 4 (Part 3): Why does running out of VRAM cause a cliff, and why can Mac act as a fallback but is slow.

Why I ran `26B A4B` first

The reason is actually quite basic: it’s about hardware reality. While 31B is certainly powerful, and the official leaderboards and initial community feedback have been very strong. However, if you run it on a machine like a 3060 12GB, the issue immediately shifts from “Is it powerful?” to “Is it worth waiting for?”. Once the model and cache start offloading to system memory, the speed can easily collapse. I will cover this in detail in a third article. 26B A4B is different. Although its total parameters are 25.2B, only about 3.8B are actually activated per token. Simply put, it’s the one in Gemma 4 that feels most “designed for local users.” So, if your machine is similar to mine—an older consumer-grade card—here is a straightforward way to decide:

If you want to see benchmark scores, go with 31B.
If you plan to use it locally in the long term, start with 26B A4B.

The Five-Point Star Problem: Someone Finally Understood My Trap This Time

I have always had a rather basic test question: asking the model to write a piece of C++ code that outputs a five-pointed star to the console.

This problem might look like a joke, but it’s actually quite tricky. Many models tend to interpret it as a pure mathematical drawing problem, and then they start talking about coordinates, trigonometric functions, and loops, ultimately outputting a mess of characters in the plain text console that is completely unreadable.

Last year, many small-parameter open-source models failed at this point.

My first reaction to Gemma 4 this time was actually quite surprising. It didn’t rush to pretend it understood; instead, it first identified the constraints and provided this judgment:

Since drawing a five-pointed star with precise geometric structure directly using mathematical logic in a plain text console (Console) is very complex (involving coordinate system transformation and pixel filling), the most classic and visually effective method is to use ASCII Art.

Regarding the Five-Pointed Star Problem, Someone Finally Understood My Trap This Time

To put it plainly, they first understood the environmental constraints behind the problem. The console is not a canvas, and the character grid is not a pixel grid. You must first figure out “how to stably output a five-pointed star,” before discussing mathematical drawing. Then, in its first version, it directly provided a hardcoded string for the five-pointed star. This action was very on point. It wasn’t about showing off derivations; it was about getting the problem solved correctly first.

What surprised me even more is that it could continue further.

If it had only stopped at ASCII Art, this problem would have only shown that it recognized the trap. What really impressed me was that when I continued to ask it to perform mathematical calculations afterward, it didn’t falter; instead, it was able to proceed logically, mapping the geometric relationships onto a character grid and finally calculating the pentagram. This demonstrates not just “it can write some code,” but rather that it understands this problem has two layers:

The first layer: What is the most stable answer for the console?
The second layer: If you insist on doing calculations, how do you reduce a geometric problem onto a character grid? Previously, many local small models would jump straight to the second layer and fail at the first. Gemma 4 reversed this approach this time; it first identified the boundaries and then decided on the solution method. I think this is more valuable than any single benchmark score.

This Coding Improvement Isn’t Just About Being “Smarter”

The reason this five-star problem is so useful is that it doesn’t just test syntax. What it truly tests is:

The ability to first understand the output environment.
The ability to admit when an intuitive solution is inappropriate.
The ability to switch between achieving “optimal presentation effect” and fulfilling “user-mandated calculation.” Once a model can solve this type of problem correctly, it indicates that the model is starting to act more like a development assistant capable of handling real-world constraints, rather than just one that completes code snippets. This is also why my first impression of Gemma 4 is much better than last year’s batch of smaller open models. Many models from last year were good at chatting, completing, and getting by, but when faced with a problem that had even slight boundary conditions, they tended to show their limitations. At least Google has addressed this weakness this time.

Translating this line cannot simply be stated as “Gemma 4 completely replaces everything”

You brought up a very key point earlier: previously, people often used Gemma for local translation. The transition to Gemma 4 isn’t actually that linear. This is because Google released TranslateGemma separately in February 2026, and it was built on the architecture of Gemma 3. What does this mean? It means that if your existing local translation pipeline is already working smoothly, you don’t necessarily have to switch everything over to Gemma 4 in the short term. Especially for scenarios with very specific goals—like only needing stable multilingual conversion—a dedicated translation model still has its value. However, if what you want is a single local model that can reasonably handle translation, Q&A, code, and general text tasks, then a more versatile route like 26B A4B is smoother. It might not be the most specialized one, but it’s more like choosing the “good enough main model to get running first” option in a real-world scenario.

Why I Don’t Want to Keep Praising `31B` in the Second Article

It’s not that 31B is bad; quite the opposite, it’s too good, which makes it easy to get distracted. If you keep focusing on the leaderboard performance of 31B, it’s easy to write this article as “Strong models are truly strong.” But what local deployment fears most is exactly that kind of talk. Because what truly determines whether you will continue using it every day isn’t the leaderboard, but rather:

Is the startup too slow?
Does the response speed drop severely?
Does long context quickly ruin the experience?
Can your own machine actually handle it? On a machine like the 3060 12GB, these practical issues are much more important than the leaderboard. So, my conclusion for the second article is simple. 31B is worth looking at; 26B A4B is worth using. For local players, these two statements are not the same thing.

My Initial Local Conclusion

If I had to summarize my experience from this test in one sentence, it would be: Gemma 4 finally feels like a local model that understands context/scenarios. Especially the 26B A4B. It might not be the model best for showing off on leaderboards, but under real-world constraints—like older hardware, consumer-grade GPUs, and long-term local use—it actually feels more like the true workhorse choice. At least with this five-star test, Google has passed.

References

Writing Notes

Original Prompt

$blog-writer Google has released the Gemma4 model after a year. As usual, I'm trying to deploy it locally on that old desktop with an unupgraded NVIDIA 3060 12GB graphics card. This time I caught the initial release, but I couldn't find an upgraded version of the commonly used Gemma3. However, there is a similar version called GemmaE4b. Please search and introduce all the models released this time, what the abbreviation letters mean in them, and then search for online reviews about Gemma4. The key point is that Google updated the model's protocol this time, and the restrictions for users are fewer. The biggest surprise: my usual test question—write a piece of C++ code to output a five-pointed star in the console. Last year's smaller open-source models couldn't handle this problem, but Google managed it this time. In the first version, it gave an answer that completely exceeded my expectations; it knew about my trap. Outputting a five-pointed star to the console is very tricky, so it directly hardcoded a string for the five-pointed star, which was outputted directly to the console. This is the original text: Because drawing a five-pointed star with precise geometric structure using mathematical logic in a pure text console (Console) is very complex (involving coordinate system transformation and pixel filling), the most classic and visually best method is to use ASCII Art. After I forced it to perform calculations, it also succeeded through mathematical calculation, successfully drawing the five-pointed star. Previously, I often used Gemma4 for local translation tasks; many multilingual versions of historical articles on current blogs are like this. The model used for local testing: gemma-4-26b-a4b. The 31b version is indeed too slow. But looking at the reviews, the 31b effect is very good, and its ranking performance is excellent. Also, while browsing forums, I realized that if the VRAM is insufficient and the model parameters are increased, the token generation speed will drop drastically. Can you explain why? Macs don't have this problem because they use unified memory; please explain the technical reason. Furthermore, if speed is required, only an NVIDIA card with large VRAM will do. The Mac solution can serve as a fallback, but it cannot match the speed. This content is very extensive; please evaluate whether it should be split into a series of articles.

Writing Outline Summary

The second article will only retain the local experience, and will no longer summarize the first article or explain VRAM principles for the third one.
First provide the hard judgment on “why run 26B A4B first,” then expand with the five-star test.
The five-star question is treated as the main axis because it better illustrates the boundary sense in coding scenarios than benchmark scores.
The translation task will be given its own section to avoid making Gemma 4 seem like a linear successor to all previous processes.

Google has released Gemma 4 this time (Part 1)

Wed, 08 Apr 2026 23:48:20 +0800

On the day of the initial release, what I originally wanted to do was simple: find an upgraded version corresponding to Gemma 3 and download it to run. However, after looking around, I was a bit stunned. The familiar naming convention of 4B / 12B / 27B is gone; instead, we have E4B, 26B A4B, and 31B. How should I put it? This time, what Google truly changed wasn’t just the model sizes, but even “how you should understand this batch of models.”

I’ve broken down these articles into three parts. This current article only clarifies the release information, model names, and protocols; the next one will cover Google Released Gemma 4 (Part II): Running Locally on a 3060 12GB, 26B A4B is More Realistic; and the last one will conclude with Google Released Gemma 4 (Part III): Why VRAM Insufficiency Causes a Cliff, and Why Mac Can Be a Fallback But Isn’t Fast.

Let’s first clarify what was actually released this time

Last year, Gemma 3 was released on March 12, 2025, and this Gemma 4 was released on April 2, 2026. It is indeed about a year apart. However, we cannot approach this by asking, “Who is the next generation after 27B.” The four main sizes provided by the official source are no longer simply categorized by total parameters. | E2B | Dense | 2.3B effective, 5.1B including embeddings, 128K context | On-device, ultra-lightweight local |

Clarify what was actually released this time

Model	Structure	Key Numbers	Typical Scenarios
`E4B`	Dense	4.5B effective, 8B including embeddings, 128K context	The original 4B small model main line

Clarify what was actually released this time

Model	Structure	Key Numbers	Typical Scenarios
`26B A4B`	MoE	25.2B total, approx. 3.8B active, 256K context	Consumer GPUs, local deployment, balancing quality and speed

Clarify what was actually released this time

Model Size	Structure	Key Numbers	Typical Scenarios
`31B`	Dense	30.7B dense, 256K context	Aiming for the upper limit, leaderboards, and more stable quality

Let’s clarify what was actually released this time

If you only look at the surface, you might feel that the naming is more confusing. But it’s not random; Google is deliberately splitting the three tracks:

Small model for on-device use, given to E2B / E4B
Local player track, given to 26B A4B
Quality and upper limit track, given to 31B This is also why many people’s first impression might be, “The previously familiar upgrade path has been broken.” It’s not that they didn’t release an upgraded version; it’s that Google doesn’t want to sell products based on just one dimension: total parameters.

‘E’ and ‘A’ are not decorative letters this time

In this batch of names, the most confusing ones are E4B and A4B. The ‘E’ in E2B and E4B stands for effective parameters, according to the official documentation. Because these two models use Per-Layer Embeddings, the total parameter count and the actual effective parameter count are not measured by the same metric. Simply put, Google is reminding you that this is not like the old “a simple 4B dense model.” The ‘A’ in 26B A4B stands for active parameters. The total size is 25.2B, but only about 3.8B are actually activated per token. This is key to the MoE approach: the total model size is large, but the part that actually participates in computation at runtime is much smaller. So, even though both names seem to have a ‘4B’, their meanings are completely different:

E4B is for the small model line.
26B A4B is a large MoE with “activation scale of around 4B” during local inference. This naming convention was indeed awkward at first, but it is closer to the actual deployment experience than before.

If you previously used Gemma 3, how to find the corresponding relationship this time

I think the easiest place to misjudge with this generation is to treat it as a linear upgrade from Gemma 3. If you look at it based on usage habits, you can roughly understand it like this:

Those who used to focus on 4B for light tasks should now first look at E4B
Those who used to focus on 27B to see the model’s upper limit should now look at 31B
If you previously wanted to find a balance point on consumer-grade GPUs that is “powerful enough but not completely unrunnable,” now focus on 26B A4B If you don’t clarify this layer first, local deployment will easily go wrong later. You might complain, “Why isn’t there the familiar upgraded version?” while mistakenly choosing a model that isn’t actually suitable for you.

The most valuable update this time isn’t the parameters

What really made me feel like this release was a “finally figured it out” moment wasn’t the leaderboard, but the license. The old Gemma terms weren’t unusable, but they always felt a bit awkward. Especially if you care about these things:

Redistribution
Distillation or secondary packaging
Integrating the model into your own product pipeline
Commercial deployment You always have to go back and look at how those notices, downstream restrictions, and accompanying agreements in the terms should be handled. By changing directly to Apache 2.0 this time, Gemma 4 made things much cleaner. The core message is very clear:
Commercially usable
Modifiable
Redistributable The main obligations are limited to retaining familiar open-source elements like the license, notices, and modification documentation. Simply put, Google didn’t just open-source a model this time; they smoothed out the entire process of “whether or not people feel safe using it.”

Initial Community Feedback, Basically Two Lines

If you only look at the first week’s buzz, there are roughly two main sentiments.

The first line is that 31B is genuinely capable. The official benchmarks are already very impressive. In the Arena AI text leaderboard, 31B was ranked among the top open-source models upon release, and it also showed a significant improvement over Gemma 3 27B on LiveCodeBench v6. Many people’s first reaction is that achieving this level of performance with this size is quite beyond expectations.

The second line is that 26B A4B seems like a lifeline for local users. It might not be the flashiest flagship model at first glance, but it is very practical. Especially if you aren’t running things in a data center, but rather on consumer-grade GPUs, workstations, or even older machines, the local experience tends to fall onto this line.

Of course, there’s a very realistic prerequisite for the initial wave of feedback: the ecosystem is still catching up with the versions. Templates, quantization methods, inference frameworks, and front-end tools—many haven’t fully kept pace yet. Therefore, when looking at comments right now, it’s best to view them in two layers:

The core model: There has indeed been a big improvement here.
Local experience: This will continue to be influenced by the maturity of the toolchain.

My Conclusion on the First Article

If you just want to know what Google actually released this time, one sentence is enough. Gemma 4 is no longer following the old idea of “a line of dense models from small to large,” but rather separating three paths: device-side deployment, local deployment, and quality ceiling. The names like E4B, 26B A4B, and 31B sound strange, but behind them is a very practical division of labor for deployment. But if you ask me what the biggest change is this time, I still stick to that judgment: It’s not about parameters, nor is it about leaderboards; it’s that Google finally put Gemma 4 into an open-source protocol that everyone feels more comfortable actually using. This step is more important than the numbers on the surface. In the next article, I won’t continue talking about the conference narrative; I’ll go straight back to local machines. Still with that unupgraded RTX 3060 12GB, why was it that my initial focus wasn’t on 31B, but on 26B A4B.

References

Writing Notes

Original Prompt

$blog-writer Google has released the Gemma4 model after a year. As usual, I'm trying to deploy it locally on that old desktop with an unupgraded NVIDIA 3060 12GB graphics card. This time I caught the initial release, but I couldn't find an upgraded version of the previously used Gemma3. However, there is a similar version called GemmaE4b. Please first search and introduce all the models released this time, what the abbreviation letters mean in them, and then search for online reviews about Gemma4. The key point is that Google updated the model's protocol this time, so the restrictions for users are fewer. The biggest surprise: my usual test question—write a piece of C++ code to output a five-pointed star in the console. Last year's smaller parameter open-source models couldn't handle this problem, but Google managed it this time. In the first version, it gave an answer that completely exceeded my expectations; it knew about my trap. Outputting a five-pointed star to the console is very tricky, so it directly hardcoded a string for the five-pointed star, and the console outputted it directly. This is the original text: Because drawing a five-pointed star with precise geometric structure using mathematical logic in a pure text console (Console) is very complex (involving coordinate system transformation and pixel filling), the most classic and visually best method is to use ASCII Art. After I forced it to perform calculations, it also managed it through mathematical calculation and successfully drew the five-pointed star. Previously, I often used Gemma4 for local translation tasks; many multilingual versions of historical articles on current blogs are like this. The model used for local testing: gemma-4-26b-a4b. The 31b version is indeed too slow. But looking at the reviews, the 31b performs very well, and its ranking scores are excellent. Also, while browsing forums, I realized that if the VRAM is insufficient and the model parameters increase, the token generation speed will drop drastically. Can you explain why? Macs don't have this problem because they use unified memory; please explain the technical reason. Furthermore, if speed is required, then an NVIDIA card with large VRAM is still necessary. The Mac solution can serve as a fallback, but it cannot match the speed. This content is quite extensive; please evaluate whether it should be split into a series of articles.

Writing Outline Summary

The first article will only focus on clarifying “what was actually released this time” and “why the protocol is important,” avoiding topics that compete with local experience discussions.
We will separate the model roadmap breakdown and then explain the meaning of the letters, making the logical flow more direct than the previous version.
For the protocol section, we retained the judgment: “What was truly released this time is not the parameters, but the usage restrictions.”
Community feedback will only be used for synthesis/conclusion, without preemptively including too many local experience details.

Writing an AI blog post, in the end, still needs to be turned into engineering (Part 3)

Fri, 03 Apr 2026 21:06:02 +0800

After going through all the configurations in the repository, I am even more certain about one thing: what matters in the end is not how strong any single model is, but rather who should bear the cost at each layer.

The most obvious signal is that the currently active published.runtime.json is still the one generated on April 2, 2026, for minimax-m2, yet the entry from April 3, 2026, at 16:38, labeled 5f17088, has switched the default provider for blog-style-suite to the local gemma-4-26b-a4b in LM Studio. This might look inconsistent, but it actually isn’t; it precisely illustrates that this pipeline has begun to specialize.

With these articles, the first two have laid out the boundaries. The first article discusses why blog-writer emerged, and the second article discusses how blog-style-suite separates style learning from token costs. This final article settles on the most practical question: where should local models, online models, and Minimax ultimately be placed?

Training Style Data, Not Worth Burning Online Models at Every Step

The issue of style data, once you start taking it seriously, quickly becomes a practical problem with tokens. It’s not about whether you want to save costs; if you don’t divide the labor, this whole setup won’t run for long. The most common mistake in the past was letting one online model handle everything.

Scraping historical articles
Performing filtering
Doing categorization
Scoring
Sampling
Enforcing style
Finally writing the draft The biggest problem with doing it this way isn’t that “the model isn’t strong enough,” but rather that every step burns the same level of cost. Looking back now, the truly reasonable approach should be to think in reverse: which steps must be online, which steps should ideally be localized, and which steps shouldn’t even be given to a model at all. As long as this boundary isn’t clear, no matter how powerful the model is, it will just end up helping you repeat a bunch of tasks that could have been pre-processed away.

Local Models are Better Suited for Dirty, Heavy, and Iterative Tasks

I am increasingly inclined to define local models as the “physical layer” for production use. They might not be the strongest, nor perfect every time, but they are particularly suited for tasks such as:

Building through repeated runs/iterations
Multi-round compression experiments on style data
Re-scanning after configuration changes
Low-risk recalculation on existing structures

These types of tasks share a clear commonality. The value isn’t in a single, extremely high-value output, but rather in the ability to run repeatedly, tolerate errors, and ideally avoid paying high costs every single round. Currently, scripts/blog-style-suite/config.json has switched to lm-studio-gemma4, which itself indicates a shift in judgment. It’s not that local gemma is necessarily stronger than online models, but for the production pipeline, we are finally starting to prioritize “runnability, frequency of use, and ability to iterate/modify repeatedly.” This point actually aligns with the logic I wrote previously in Don’t force strong tasks onto weak models. Local models might not be suitable for writing complex, comprehensive articles from scratch, but they are excellent for handling dirty, heavy, and batch processing tasks. Preprocessing style data is inherently more like this category of task.

Online models are better suited for the final polish, not for doing everything from scratch

Just because local models are suitable for the production side doesn’t mean online models have no value. The real value of an online model lies precisely in that final polishing touch. For example:

Supplementing facts based on the latest information
Structuring arguments within a larger context
Handling time-sensitive information that requires internet verification
Transforming already prepared structured style assets into a publishable article These tasks require higher demands on expression quality, factual integration, and contextual understanding, making online models more valuable here. In other words, the powerful model is more like the final few assembly line steps. It’s not that it can’t do more upfront work, but if you make it scan from beginning to end, the entire cost structure will quickly become distorted. This is also why blog-writer is designed to only read from the published location published.runtime.json, rather than having to switch providers or re-scan the suite directory while drafting. The lighter the consumption side, the better it is for a more powerful model to focus on finalizing the article.

The Significance of Minimax: It’s Not Just Another Provider Connection

Many people who see Minimax might first think: “It’s just another model being connected.”

I don’t think so.

The truly valuable aspect of Minimax is that it has successfully paved the way for “multiple provider outputs consumed by a single publishing contract.”

The change on April 2, 2026, at 10:18 (9f15199) modified blog-style-suite to support multi-model configurations, with outputs isolated per provider. Subsequently, the README and runtime structure have consistently emphasized one thing: while the suite can generate many sets of results, only the manually selected published.runtime.json is actually effective.

This boundary is extremely important.

Because once this boundary is clear, the role of Minimax changes from being “something that must be bound within the drafting process” to becoming:

Something that can participate in production-side comparisons.
Something that can be used to generate a runtime version.
Something that can be compared horizontally with local model artifacts.
Finally, something whose publication is decided by human judgment.

This transforms the provider from a “system dependency” into a “replaceable component.”

I believe this is the most interesting significance of Minimax within this engineering setup. It isn’t here to dominate the entire pipeline; it’s here to validate whether this pipeline has successfully cleaned up its interfaces.

True specialization is not based on model strength, but on task type

I now favor a classification method that is quite rudimentary, but very effective.

Rules and Hard Constraints

Leave to local scripts. If it can be solved with deterministic tools like scanner.py, write_post.py, or write_post_series.py, don’t let the model get involved.

Style Data Generation

Prioritize local models or lower-cost providers. Because what is most important here is reproducibility, room for iteration/error, and cacheability, not necessarily the most dazzling single output.

Final Drafting and Fact Consolidation

Hand this off to a model better suited for long-context integration, expression consolidation, and fact-checking/web retrieval. This layer is where spending money on online models is most worthwhile. When broken down like this, many previously confusing issues are actually not that complex. You don’t need to argue every day about “which model is the strongest”; you just need to ask: which layer does this task belong to?

Ultimately, what is most valuable is not the model, but the clear boundaries.

This concludes my third article. As blog-writer and blog-style-suite have evolved, I feel that what is most valuable is not which provider we connected next, or who we replaced, or which one we tested. What is most valuable is that the boundaries are finally becoming clearer.

blog-writer handles the consumption side.
blog-style-suite handles the production side.
published.runtime.json is the publishing point.
Local models are better suited for dirty and heavy lifting that needs to be run repeatedly.
Online models are better suited for the final polish/wrap-up.
Online providers like Minimax feel more like replaceable components rather than the central hub of the system. Once the boundaries are clear, the entire workflow flows smoothly. You won’t expect one model package to conquer everything, nor will you stack every step onto the most expensive layer. In the end, while it looks like selecting a model, what we are actually doing is assigning workstations for different types of tasks. Simply put, having a single strong point is certainly good. But in the long run, clear boundaries are often more important and stronger than any single point solution.

References

Repository Commit: 9f1519967981c5eef7bd1eb407b0406ac542ebd0
Repository Commit: 5f17088391ee858b88fc50df884bc0103ff0b3c1
Repository File: scripts/blog-style-suite/config.json
Effective Runtime: .agents/data/blog-writing/published.runtime.json
Related Old Article: A Period of Heavy AI Programming
Related Old Article: Ultimately Returning to Domestic Models
Related Old Article: Don’t Force Strong Tasks with Weak Models

Writing Notes

Original Prompt

$blog-writer This content is quite extensive, so I've split it into a series of articles: Last year, many drafts were written using large models. Back then, the process was to create an outline or a list of questions myself, and then have the AI generate the draft, copy the content into a local md document, fill in header information, tag information, and publish the article; recently, I used Codex a lot and found that its web search capability is very strong. So, could I write a skill to automate these tasks? This led to the first draft of the skill blog-writer. I also thought about having the AI learn my previous writing style, which caused blog-writer to consume a lot of tokens when running. Subsequently, I optimized blog-writer in several versions, splitting out the data module and the data generation module. The original data generation module was still an independent skill. As I continued writing, I realized that it would be better as a Python project, which led to blog-style-suite. Then, I found that training on style data also consumes a lot of tokens, so I wanted to use a local large model and connected to a local LLM. I then thought about comparing the differences between the local LLM and the online version, so I integrated minimax; the evolution history of blog-style-suite and blog-writer can be analyzed from the git commit history. Additionally, based on the code for local blog-writer and blog-style-suite, I can discuss the design ideas, how token saving was achieved, and how the data structure was designed—the core design concepts. If tokens are abundant, it can consume entire historical articles; preprocessing can save a lot of tokens.

Writing Strategy Summary

The third article will no longer repeat the discussion on architecture, but instead focus solely on the practical issue of “model specialization/division of labor.”
Start directly by stating the current reality—whether to use published.runtime.json from the current repository or if it’s switched locally to gemma4 via minimax-m2 or config.json—to reduce filler content.
The focus should not be on proving which model is stronger, but rather on explaining why different tasks should be assigned to different cost layers.
Placing Minimax in the “replaceable provider” section aims to pull its significance back into the engineering boundary, rather than treating it as just another entry on a model leaderboard.
Conclude by returning to the overarching judgment: “Clear boundaries are more important than single points of strength,” serving as the closing statement for the entire series of articles.

Making the "AI writes blog" thing into an engineering project later (Part II)

Fri, 03 Apr 2026 21:02:02 +0800

If there are enough tokens, the least effort method is actually quite crude: just feed the model historical articles and let it learn on its own. The problem with this method is that it only suits occasional writing, not continuous work. If you treat blogging as a long-term workflow, relying solely on raw historical articles will quickly go from “simple and direct” to “expensive and messy.”

With these articles, the main thread has shifted. The previous article, AI Writing Blogs: It Eventually Needs to Become an Engineering Process (Part 1): Why a blog-writer is inevitable, discussed automation on the consumption side. This article starts discussing the production side—how to generate style data, how to compress it, and how not to waste tokens; the next article will continue this in AI Writing Blogs: It Eventually Needs to Become an Engineering Process (Part 3): Local Models, Online Models, and Minimax—How They Finally Divide Labor.

The most natural initial thought is to just feed it historical articles.

This path feels too natural. If you want the model to learn your writing style, the most intuitive way is certainly to feed it old articles. It’s best to include all the posts from your history that are most like your own, and let it summarize them itself. For a single task, this approach has no flaws. In fact, many times the results are quite good. If the context is long enough, the model is powerful enough, and there are enough historical articles, the style can indeed be captured. But the problem isn’t “can it write this one article,” the problem is “for the next one, the one after that, do we have to repeat this process?” Feeding a new batch of old articles every time brings several very practical side effects:

The same batch of material repeatedly occupies the context window.
Token overhead grows almost linearly with the number of drafts written.
The model sees more and more noise, causing genuinely useful signals to become diluted.
The drafting action and style maintenance action become completely bound together; neither can be easily reduced. In other words, when tokens are abundant, eating it raw certainly works. But from an engineering perspective, we cannot keep doing that forever.

This is also why the data module and the data generation module must be separated

I later realized that the core idea can be summarized in one sentence: separating the consumption side from the production side.

blog-writer is responsible for the consumption side. It only reads an already published runtime and then writes out the article according to a fixed contract.

Meanwhile, scanning, filtering, scoring, compressing style data, and provider comparison—all of this should be placed in another production pipeline. This is what later became blog-style-suite.

Looking at the git history, this turning point is very clear.

The commit 84a06b5 on April 1, 2026, at 21:47 clearly replaced the original blog-style-maintainer skill with a repository-level CLI tool. This action speaks volumes because once you have scan/build/rebuild, an output directory, and a recovery mechanism, it’s no longer like a simple skill; it’s more like a normal Python project.

By the commit 9e92b8e on April 1, 2026, at 23:05, blog-style-suite was further broken down into modules like scanner.py, builder.py, and compressor.py. By this stage, the thinking process was already highly engineered:

scanner.py is responsible for scanning articles from disk and extracting structured features.
builder.py is responsible for scoring, selecting, caching, and runtime assembly.
compressor.py is responsible for the compression steps that involve the model.

This represents a completely different approach compared to simply writing a super prompt.

Saving Tokens: Not by Magic, But by Preprocessing and Batching

The most valuable part of this entire engineering setup, I think, is the commit bc4b950 from April 2, 2026, at 19:41. That commit was very direct: it reduced AI calls from about 2000 times down to a maximum of 5 times per provider. How was this achieved? It wasn’t by “making the prompt smarter,” but by doing the necessary preprocessing beforehand. The current flow in blog-style-suite is very clear:

The scan stage is purely heuristic, requiring 0 AI calls.
The build stage first performs heuristic scoring, also requiring 0 AI calls.
Then, it performs one batch selection and labeling for each of the four lanes: technical / finance / essay / tooling.
Finally, there is one author style compression step. Counting this up, the cold start requires at most 5 calls. More critically, these 5 calls are not spread across every single article; they are concentrated on high-value summary materials that have already been preprocessed. This is where preprocessing truly saves tokens. It’s not about saving a few words; it’s about changing the process from “calling per article” to “batch calling by stage.” Furthermore, caching has been implemented. In builder.py, there are lane batch fingerprints, provider checkpoint recovery, and contractions like review_pool_per_lane = 12 for local model context. If you change a small amount of data, the entire pipeline doesn’t need to rerun. These kinds of designs might not look flashy, but every single one is highly practical because they solve the problem of “don’t let the same batch of tokens burn twice.”

Essentially, this data structure is compressing the truly useful signal.

Once this is broken down, the data structure will be smooth. I am now more willing to understand it as three layers.

Layer One: `scan.json`

This is the shared raw material. It contains structured signals such as article path, title, date, category, tags, opening paragraph, closing stub, headings, screening results, and lane classification. It is not directly consumed by blog-writer; rather, it is passed to the production side for further processing.

Second Layer: `{provider}.source.json`

This is the provider-level checkpoint. Building upon the shared raw materials, it includes intermediate states such as scoring results, lane selection, fingerprint, and cache status. In other words, it is more like a “semi-finished product during processing,” with an emphasis on being recoverable, reusable, and resumable.

Layer Three: `{provider}.runtime.json` and `published.runtime.json`

This is what the consumption side truly cares about—the finished product. It retains:

author_style
lanes
samples
writer_guide In essence, it compresses a large collection of historical articles into one ready-to-consume runtime style asset. The published.runtime.json in particular is crucial for the publishing stage. The blog-writer only reads this file and does not need to scan content/post, nor does it need to care about the complete images of all providers in the suite directory. Once this boundary is established, the consumption side becomes much lighter. The writing model no longer sees a pile of raw old articles, but rather a pre-processed, high-density signal.

Not Everything Should Be Left to the Model

I’m increasingly feeling that the most correct judgment in this entire engineering process isn’t “add more models,” but rather “don’t throw tasks that shouldn’t be done by a model onto it.” Things like these are much better handled by local rules first:

Frontmatter parsing
Extracting introductory paragraphs
Headings extraction
Determining author/repost/model attribution
Detecting blockquote ratios
Hard rule filtering for things like , embedded prompts, and body length. Having the model do these tasks isn’t impossible, but it’s wasteful. What models are better suited for are parts that involve ambiguity or trade-offs. For example, which few articles in a lane best represent the current sentiment, or extracting author style tags from high-scoring articles. Therefore, what makes blog-style-suite truly valuable isn’t just “saving tokens,” but rather its re-division of labor among humans, rules, and models—assigning each party the tasks they are best suited for.

Preprocessing isn’t about saving a few tokens; it’s about making the act of writing sustainable.

For the conclusion in the second article, I want to make it more direct.

When you have plenty of tokens, reading historical articles raw is fine. In fact, if you only write one or two pieces, it might even be less mentally taxing.

But as soon as you want to turn this into a long-term workflow, preprocessing becomes non-negotiable. Because without preprocessing, the writing model has to re-read old materials every time, and style maintenance and article generation are always mixed together.

The significance of blog-style-suite is to untangle this mess.

It’s not about making the system look complex, nor is it just for another project name; it’s so that blog-writer can remain lightweight, stable, and focused on “only the action of writing.”

Having reached this point, the next question naturally follows.

Since the production side has been separated, what model should bear this cost? Local models, online models, or Minimax—where should each one stand in the workflow? I’ll save this for the next article: AI Writing Blogs: How It Eventually Has to Become Engineering (Part 3): The Division of Labor Between Local Models, Online Models, and Minimax.

References

Repository Commit: 84a06b5dc743f2e9bc6e788d53496a1261bc63ae
Repository Commit: 9e92b8e6a15d03e6392aff7f3b2dcb0992fe5043
Repository Commit: bc4b950cbb13e37d1fdb16a9d23325cfefa6f90e
Repository File: scripts/blog-style-suite/README.md
Repository File: scripts/blog-style-suite/style_pipeline/scanner.py
Repository File: scripts/blog-style-suite/style_pipeline/builder.py
Repository File: scripts/blog-style-suite/style_pipeline/compressor.py
Effective Runtime: .agents/data/blog-writing/published.runtime.json

Writing Notes

Original Prompt

$blog-writer This content is quite extensive, so I've split it into a series of articles: Last year, many drafts were written using large models. Back then, the process was to create an outline or a list of questions myself, and then have the AI generate the draft, copy the content into a local md document, fill in header information, tag information, and publish the article; recently, I used Codex a lot and found that its web search capability is very strong. So, could I write a skill to automate these tasks? This led to the first draft of the skill blog-writer. I also thought about having the AI learn my previous writing style, which caused blog-writer to consume a lot of tokens when running. Subsequently, I optimized blog-writer in several versions, splitting out the data module and the data generation module. The original data generation module was still an independent skill. As I continued writing, I realized that it would be better as a Python project, which led to blog-style-suite. Then, I found that training on style data also consumes a lot of tokens, so I wanted to use a local large model and connected to a local LLM. I then thought about comparing the differences between the local LLM and the online version, so I integrated minimax; the evolution history of blog-style-suite and blog-writer can be analyzed from the git commit history. Additionally, based on the code for local blog-writer and blog-style-suite, I can discuss the design ideas, how token saving was achieved, and how the data structure was designed—the core design concepts. If tokens are abundant, it can consume entire historical articles; preprocessing can save a lot of tokens.

Writing Outline Summary

This article shifts the focus from the act of writing drafts to data engineering, with the core answer being “why modularization is necessary.”
The introduction directly acknowledges that “using raw historical articles works,” which makes the subsequent arguments for splitting more convincing.
It elaborates on the three structural layers: scan.json, source.json, and runtime.json, avoiding vague architectural discussions.
bc4b950 is placed in the middle as a turning point because “reducing from about 2000 times to 5 times” best illustrates the value of preprocessing.
The conclusion re-separates the consumption side and the production side, setting the stage for model division in the third article.

AI Writing a Blog: The Next Steps Towards Engineering (Part 1)

Fri, 03 Apr 2026 20:58:02 +0800

I wrote quite a few AI articles last year. The most basic workflow back then was: first, organize an outline or a list of questions myself; let the large model spit out the main body text; then copy the content into a local md document, add frontmatter, tags, categories, and titles, and finally publish it. This process isn’t unusable, but it’s tedious. The part that really wastes time isn’t the main body text, but the repetitive labor surrounding it. Especially after using Codex a lot recently, this awkwardness has become even stronger. It can read repositories, modify files, supplement materials, and even write articles directly into the directory. If I still have to copy and paste things manually, it feels like I’m tying down the tool’s legs.

This series of articles is actually trying to convey one thing: AI writing blogs cannot rely solely on a single prompt in the long run. This current article first discusses why blog-writer came into existence; the next article will continue with AI Writing Blogs: Later, It Still Needs to Be Engineered (Part II): How blog-style-suite Separates Style Learning and Token Costs; and the last article concludes with AI Writing Blogs: Later, It Still Needs to Be Engineered (Part III): How Local Models, Online Models, and Minimax Will Finally Divide Labor.

What’s truly annoying isn’t writing the draft, but that sequence of mechanical actions.

The early workflow was essentially like an outsourced assembly line. I would first list out the problems clearly, or build a rough outline. The model is responsible for laying out the main body text. Then, a human comes back to complete the remaining publishing steps.

Copy to local md file
Fill in title, date, and slug
Add tags and categories
Insert 
Organize reference materials
Finally, decide which directory it should go into Looking at each step individually, this sequence isn’t difficult. But when strung together, it becomes tedious. What’s annoying isn’t the technical difficulty; it’s that these steps are all mechanical, yet they cannot be skipped. This is why I increasingly feel that changes like AI coding interaction based on command line are not just about “changing the entry point.” When AI can directly read and write files within a repository, if blog writing still stops at the level of “copying the body text to a local document,” the entire workflow is actually outdated.

blog-writer The First Layer of Value: It’s Not the Style, It’s Locking Down the Contract

The very first node for blog-writer was at 17:00 on April 1, 2026, with the commit hash 991536a. Looking at the git commit history, this version included SKILL.md, write_post.py, and an initial set of style guidelines all together.

However, when I looked back later, the most valuable part of this draft wasn’t that “the AI learned my writing style,” but rather that it established a rigid contract for content creation.

What does “locking down the contract” mean?

The input must include at least an outline and factual anchors.
The output must be complete Markdown, not a work in progress.
Frontmatter cannot rely on manual additions anymore.
The article cannot just stay in the chat window; it must land directly into content/post.

This point is crucial because prompts themselves are inherently unstable. If you say, “Write it like before,” today, it might understand that only the tone should be similar; if you repeat it tomorrow, it might only learn superficial sentence structures. But once it’s written as a Skill, the rule shifts from “improvisation” to a “fixed workflow.”

The subsequent nodes were actually all about reinforcing this contract.

The commit at 22:54 on April 2, 2026, with hash 8eb735a, standardized elements like author fields, writing notes, and original prompts. By this stage, the blog draft was no longer considered “finished once the body text is done”; instead, metadata, traceability, and public notes were all standardized together.

Therefore, the first layer of value for blog-writer has never been about making the model seem better at writing; it’s about finally giving the act of drafting repeatable boundaries.

Series Mode, Which is Actually One Step Forward in Writing Workflow

After stabilizing writing a single article, the next problem quickly emerged. Some topics are simply not suitable to be crammed into one piece. If you force it, the result often becomes a long article that is information-heavy, has a scattered main thread, and fails to fully explain every point. This is why the commit 1a5604e on April 2, 2026, at 23:55 was so crucial. That time, they directly added the series mode along with write_post_series.py. The articles are linked using relref, and the replacement is done uniformly during batch writing. This might look like a minor upgrade to a file-writing script, but it’s not. It illustrates one thing: content engineering is no longer just about “how to generate this single article,” but rather starts considering “how to stably save this set of content, how to guarantee the order, and how to link between them on the site.” The next day’s commit, 04dccb9 on April 3, 2026, at 09:29, pushed this process one step further. The timestamps for series articles now increment by minutes instead of sharing a single timestamp. This change is small but very “engineering-y” because it solves real problems like Hugo list pages, previous/next article navigation, and series ordering. Simply put, the series mode isn’t about looking advanced; it’s about eliminating the need for manual fixes when publishing multiple articles together.

But relying on just one skill will eventually hit a token wall.

The problem lies here. Once you start seriously tinkering with style learning, the context of blog-writer will quickly become bloated. You not only want it to write, but you also want it to write like you used to. The most natural way to do this is to dump all your historical articles into it. This works for a single run, of course. But as soon as you’re not writing an occasional piece, but trying to make it a long-term workflow, the problems immediately arise:

High token consumption
Repeatedly feeding the same batch of old articles every time
Model attention is diluted by old material
Drafting and style maintenance are intertwined; neither is easy.

It was from here that I slowly realized that blog-writer is better suited for the consumption side, rather than trying to feed it everything. The act of drafting should be as light, direct, and limited to reading only the effective versions as possible. As for how to generate, filter, or compress style data, that’s a matter for a separate production pipeline. This realization finally pushed me to the next step, which was AI Blog Writing: It Still Needs to Be Engineered (Part II): How blog-style-suite Separates Style Learning from Token Costs.

First, stabilize the process; only then can we talk about style and models.

Looking back now, blog-writer didn’t emerge because I suddenly wanted to build a blog writing assistant. It was more because the original workflow started failing to keep up with new ways of working. Once a tool like Codex can connect to the internet for supplementary material, read and write within repositories, and directly call scripts, the act of writing a blog shouldn’t stop at “copying the body text to a local document.” If you don’t automate this part, it will actually become the clumsiest link in the entire chain. So, I’ll leave the conclusion for the first post here. What blog-writer solved initially wasn’t writing style, but the repetitive labor of the publishing action. Without this layer of contract, any subsequent discussion about tokens, data structures, or local models is actually baseless.

References

Repository Commit: 991536a237d04aba7c44dec501b3d98c644040c8
Repository Commit: 8eb735aa8448c97deb2af1ea46b86772008fa9e3
Repository Commit: 1a5604e7e6ce0a13f260fcbb8c2c1d964cdd0892
Repository Commit: 04dccb98c55a6ea3b81408012b33a6219cf8ab77
Repository File: .agents/skills/blog-writer/SKILL.md
Repository File: .agents/skills/blog-writer/scripts/write_post.py
Repository File: .agents/skills/blog-writer/scripts/write_post_series.py

Writing Notes

Original Prompt

$blog-writer This content is quite extensive, so I've split it into a series of articles: Last year, many drafts were written using large models. Back then, the process was to create an outline or a list of questions myself, and then have the AI generate the draft, copy the content into a local md document, fill in header information, tag information, and publish the article; recently, I used Codex a lot and found that its web search capability is very strong. So, could I write a skill to automate these tasks? This led to the first draft of the skill blog-writer. I also thought about having the AI learn my previous writing style, which caused blog-writer to consume a lot of tokens when running. Subsequently, I optimized blog-writer in several versions, splitting out the data module and the data generation module. The original data generation module was still an independent skill. As I continued writing, I realized that it would be better as a Python project, which led to blog-style-suite. Then, I found that training on style data also consumes a lot of tokens, so I wanted to use a local large model and connected to one locally. I then thought about comparing the differences between the local large model and the online version, so I integrated minimax. The evolution history of blog-style-suite and blog-writer can be analyzed from the git commit history. Additionally, based on the code for local blog-writer and blog-style-suite, I can discuss the design ideas, how token saving was achieved, and how the data structure was designed—the core design concepts. If tokens are abundant, it can consume entire historical articles; preprocessing can save a lot of tokens.

Writing Strategy Summary

The first article should focus on the workflow trigger point, without rushing to detail the division of labor between tokens and models, to avoid having all three articles compete for the main narrative.
It retains the key insight: “The body content is not difficult; what’s troublesome are the mechanical actions before and after publishing.”
By using nodes like 991536a, 8eb735a, 1a5604e, and 04dccb9, we ground the concept of “process contracturization” in actual Git evolution.
The series pattern is reserved for this article to illustrate that blog writing has moved from generating single pieces to managing entire sets of deliverables.
The ending deliberately points toward the token wall, setting up the groundwork for data engineering and preprocessing in the second article.

Skill is not a new prompt, it is the job manual for the agent.

Thu, 02 Apr 2026 22:43:16 +0800

These past few days, while reading about AI programming, people were first discussing MCP, and then immediately started talking about Skill. Many people who see this term for the first time will instinctively treat it as another new protocol or another advanced prompt.

My judgment is very straightforward: Skill isn’t here to replace MCP; rather, it’s more like providing an occupational manual for the agent. MCP solves the problem of “enabling the agent to connect to the external world,” while Skill solves the problem of “how to reliably get the job done after connecting.” These two are not a replacement relationship; they are more like one following the other.

Simply put, MCP gives the agent hands and feet, and Skill tells the agent not to mess around.

What exactly is a Skill?

If I were to explain it in the simplest terms, I would say this: It’s like taking the expert knowledge from a seasoned employee’s head and organizing it into a reusable, triggerable, and actionable manual. That thing is a Skill. The official OpenAI documentation defines it very directly: A Skill is a package of capabilities designed for specific tasks. It can contain instructions, reference materials, and optional scripts. The goal isn’t to make the model “smarter,” but rather to ensure that when performing a certain type of task, it outputs results consistently according to a fixed workflow. What is it most like?

It’s not like a regular prompt because it’s not something you just say once and are done with.
It’s not like an MCP because it doesn’t handle connecting tools and data sources.
And it’s not like the general rules in AGENTS.md because it’s not a universal rule for the entire repository. It is more like a specialized Standard Operating Procedure (SOP), or a trade manual. For example:
When handling GitHub PR comments, first identify which comments need attention, then ask the user which ones to handle, then modify the code, and finally provide feedback.
When debugging a CI failure, first pull the GitHub Actions logs, then extract the failing segments, then propose a fix plan, and only execute after approval.
When writing a blog post, first generate titles according to a fixed style, then supplement with facts, then add frontmatter, and finally publish it. The common thread among these tasks is not that “the model doesn’t know how to answer,” but rather that “the model’s approach varies every time, making it easy for it to drift off course.” This is where the value of a Skill comes into play.

The Difference Between Skill and MCP

This issue really needs to be viewed within a real workflow, otherwise, it’s easy to talk too theoretically.

MCP is like the interface layer. The official definition of MCP is an open standard for connecting AI applications to external systems. Files, local databases, search engines, design mockups, third-party services—all of these can be connected via MCP. Therefore, it solves the problem of “connecting things.”

Skill is like the process layer. The OpenAI Agent Skills documentation makes it very clear that a Skill is the writing format for reusable workflows. A Skill must have at least an SKILL.md, and can also include scripts/, references/, and assets/. Codex first reads its name and description; only when it determines that it needs to use it does it load the full description into the context. This is what the official documentation calls progressive disclosure.

So, the division of labor between the two is very clear:

MCP: Connects the capabilities (the “what”).
Skill: Defines the order of operations (the “how”).

To give a very intuitive example, something like converting Figma to code. If the agent cannot read the design mockup at all, then what you need first is an MCP. But if it can already read the design mockup, but just writes things randomly—writing components today, cutting pages tomorrow, and forgetting visual checks the day after—then what you need to improve is the Skill.

How to Develop Skills

This is actually not as heavy as you might think. OpenAI officially recommends starting with the built-in $skill-creator, which will help you build the basic structure, such as trigger conditions, scope, and whether a script is needed. By default, it prioritizes instruction-only, meaning don’t rush to write scripts; first, make sure your instructions are clear. If writing manually, the minimum structure is very simple:

my-skill/
├── SKILL.md
├── scripts/
├── references/
└── assets/

Of these, only SKILL.md is truly essential. Furthermore, this file must have at least two pieces of metadata:

---
name: skill-name
description: Explain exactly when this skill should and should not trigger.
---

I think the most crucial steps when developing a Skill are the following.

1. Look for “Recurring Biases”

Not every task is worth turning into a Skill. If you only ask the agent to do something occasionally, a regular prompt is enough. The situations that are truly suitable for creating a Skill are usually like this:

You have repeated the same thing many times.
The agent has the capability, but the execution order always changes.
The location where it makes mistakes is similar every time. Simply put, it’s not an “ability gap,” but rather an “inconsistent process.”

2. Write the `description` as a trigger condition, not as marketing copy

This step is crucial. The official documentation specifically emphasizes that whether Codex will implicitly call a Skill heavily depends on the description. So, don’t write things like “This is a very useful skill”; instead, write “When it should be used and when it should not be used.” For example, the description for the official gh-fix-ci skill is very clear: Use this when the user asks you to debug or fix failed GitHub PR checks; the focus is on checking logs, summarizing the failure reasons, providing a remediation plan, and only implementing it after receiving explicit approval. Just by reading it, you know its boundaries.

3. If it can be solved with instructions, don’t write a script first

OpenAI’s documentation is also very practical: unless you explicitly need deterministic behavior or external tools, prioritize using instructions rather than scripting everything right away. Why? Because the more scripts you have, the higher the maintenance cost becomes. Many Skills initially only require clearly explaining the steps:

What to do first
What to do next
What the output format should be
In which situations to stop and ask the user This can solve most of the problems. Only when a step is particularly stable, mechanical, or highly suitable for automation should you move it down into scripts/.

4. Extract Data, Templates, and Resources

As you write the Skill, it’s easy for it to become one long block of descriptive text. This is when you need to break things out.

SKILL.md is responsible for explaining rules and sequence.
references/ holds background documents and reference materials.
assets/ stores templates, icons, and examples.
scripts/ contains actions that can be executed reliably. Doing this has two benefits. First, the main file won’t become increasingly bloated. Second, the model only loads details when necessary, which aligns with the concept of progressive disclosure.

5. Local Use First, Cross-Project Distribution Later

The official documentation explains this very clearly. If you are only using it for your current repository, placing it in .agents/skills/ is sufficient. Codex scans the skills directory from locations such as the repository, user, and system. However, if you find that this functionality can be used across more than one repository, or if you want to package and distribute multiple skills together, then don’t stop at just the skill folder level; you should consider using plugin. The official OpenAI documentation is also very clear: Skill is the workflow itself, while plugin is the unit that is better suited for installation and distribution.

What Scenarios are Suitable for Skills

Having more Skills is not always better; they are best suited for the following types of tasks.

High-Frequency Repetitive Tasks

Tasks you do every week, and the sequence is pretty much the same each time. For example:

Handling PR review comments
Writing a blog post and adding frontmatter
Debugging CI failures
Performing pre-release checks What’s most daunting about these kinds of tasks isn’t that the model can’t do it, but having to explain it all over again every single time.

Tasks with Fixed Procedures

Some tasks naturally have a sequence. For example, when troubleshooting an issue, you should first check the logs, then narrow down the scope, then propose a plan, and finally make changes. In this case, Skill is particularly suitable because it can enforce the order, reducing reliance on the model’s ad-hoc performance.

Tasks Requiring Domain Context Binding

Some tasks do not have fixed steps and come with strong constraints. For example:

Must only check official documentation
Must output in a specific review format
Must retain existing terminology within the team
Must comply with the writing or development standards of a certain repository If you rely on prompts alone every time, it’s easy to miss these details.

Tasks Requiring Tools and Processes Together

This scenario is particularly typical. It’s not just about “connecting to GitHub” and being done; after connecting, you still need to examine the logs in a certain way, deduce the problems, decide whether or not to modify anything, and finally how to provide feedback. In other words, external connections and internal processes must work together. This is often when MCP + Skill appear jointly.

When Not to Use a Skill

It’s also important to be clear about this, otherwise it’s easy to want to make everything into a Skill.

One-off Casual Tasks

When a user asks a quick question, a standard prompt is often sufficient.

You just want to “connect an external system”

In that case, prioritize MCP, not Skill.

You want to constrain the long-term behavior of the entire repository

This is more like the job of AGENTS.md, not a Skill. So, to summarize simply:

Missing connections, use MCP
Missing processes/workflows, use Skill
Missing global rules, use AGENTS.md

Several Representative Examples

It’s better to look at a few real-world examples than to hear too many concepts.

1. `roll-dice`: The Minimal Viable Entry Case

This example comes from the official OpenAI Agent Skills documentation. It is very small; there is almost only an SKILL.md in the directory, which allows the agent to call PowerShell’s random number command when the user requests rolling dice. Why is this example good? Because it directly exposes the most core skeleton of a Skill:

It has clear triggering conditions
It has a clear execution method
It has clear boundaries It illustrates one thing: a Skill doesn’t have to be large. As long as something happens repeatedly, and you don’t want the model to improvise wildly, you can create it as a Skill.

2. `gh-address-comments`: A Workflow Example for Handling GitHub Comments

This example comes from the official OpenAI openai/skills repository. Its goal is not to “connect to GitHub,” but rather to encapsulate the process of “handling comments on the current branch’s PR.” The steps in the official version are very typical:

First, confirm if gh is already authenticated.
Then, fetch the comments and review threads for the current PR.
Number and summarize these comments.
Allow the user to explicitly select which ones need processing.
Only then start the actual work. This example particularly illustrates the value of a Skill. For many engineering tasks, the difficulty isn’t whether “the model knows what GitHub is,” but rather “whether it will process things in the correct order.” gh-address-comments solves exactly this kind of sequencing problem.

3. `gh-fix-ci`: Troubleshooting Engineering Cases with Failed CI

This is also the official skill in the openai/skills repository. It addresses another very typical engineering task: a PR check has failed, should it be fixed, and how to fix it. The workflow defined in this Skill is also very representative:

First, confirm the gh login status.
Find the current PR.
Pull the failed checks and logs from GitHub Actions.
Extract the failure snippets.
Propose a fix plan first.
Only act after getting approval. This scenario cannot be reliably handled by a simple prompt like “Help me see why CI failed.” Because it involves permissions, logs, external tools, approval boundaries, and execution order—all of which need to be defined.

4. Private Repository Skills: Solidifying Team’s Own Methodologies

Beyond the official examples, I think the greater value of Skill lies within private repositories. For instance, the blog-writer in this blog repository is essentially a very typical repo-scoped skill. It doesn’t aim to “teach the model how to write Chinese,” but rather codifies the writing style, structure, fact-checking process, output path, and final storage format that have already been established within this specific repository into a workflow. These kinds of Skills often hold the most practical value. This is because they are not designed for everyone; instead, they specifically solve the problem: “In this repository, what kind of task keeps recurring, and where do we tend to go off track?”

When to Actually Use a Skill

So, let’s get back to the most practical question: when should you seriously implement a Skill? My answer is: When you realize the problem is no longer “the model lacks capability,” but rather “its execution is inconsistent every time.” At this point, continuously stacking prompts yields diminishing returns. If you add one sentence today and another tomorrow, eventually the prompt becomes like a rambling diary entry, and the model will still make mistakes. Instead, organizing it into a Skill—separating the trigger conditions, steps, boundaries, scripts, and necessary data—results in more stable and reusable outcomes. The MCP brings in the external world; the Skill solidifies the internal methodology. The former solves “if it can be done,” while the latter solves “how to do it reliably.” This is my current understanding of a Skill. It’s not just a new prompt, nor is it a new protocol. It’s more like an operational manual for the agent.

References

Writing Notes

Original Prompt

Prompt: AI Large Model Programming, first appeared MCP, then Skill. Using plain language, explain what Skill is, how to develop a Skill, what scenarios are suitable for Skill, and provide specific, representative examples from each source.

Writing Approach Summary

Instead of repeating a “concept overview of MCP and Skill,” the focus is placed on the role and boundaries of Skill.
Abstract definitions are grounded in real workflows using common analogies like “trade manuals” or “specialized SOPs.”
The development section follows the actual structure of official documentation, retaining key points such as description, progressive disclosure, and instruction-only.
Case studies prioritize official sources, utilizing roll-dice from the official documentation, along with gh-address-comments and gh-fix-ci from the openai/skills repository.
The conclusion re-clarifies the boundaries between MCP, Skill, and AGENTS.md to prevent readers from remaining confused after reading.

Don't force weak models onto hard tasks.

Thu, 02 Apr 2026 22:05:00 +0800

Recently, I’ve been migrating some edge cases to MiniMax and local models. The more I use them, the more I feel that we shouldn’t always measure things by the standard of “the most powerful model.”

My judgment is straightforward: don’t force weak models into hard tasks. Models like MiniMax are indeed limited in capability, but for complex coding, long-chain reasoning, or ambiguous requirement decomposition, they fall a bit short. However, if you ask it to do data cleaning, document writing, or searching for proposal materials—these kinds of tasks—it can handle them perfectly well. The same logic applies to local models around the 12B size; translation, format rewriting, and batch cleaning are actually where they are best suited.

To put it plainly, it’s not that the models lack value; it’s just that we shouldn’t place them in the wrong roles.

The real problem isn’t how strong the model is, but whether it works correctly.

Many people who talk about large models automatically think of the most difficult tasks.

Writing complex engineering code independently
Deconstructing an entire system in one go
Multi-turn reasoning over long contexts
Planning and executing while searching These are certainly important. But in real-world work, what is actually piled on your desk most often isn’t these kinds of tasks. It’s more like:
Cleaning up a pile of dirty fields
Organizing scattered information into readable documents
Converting long texts into summaries, FAQs, or outlines
Standardizing mixed Chinese and English content formats
Gathering data from multiple web pages and then compiling it into a draft proposal For these types of tasks, what is most needed is not “the model thinking like a genius,” but three things:
Instruction following must be reasonably accurate.
Output structure should be as stable as possible.
The cost must be low enough that you are willing to use it repeatedly. This is why I always feel that weak models are not useless; they just cannot be used in the same kind of battle as flagship models.

MiniMax: What’s Actually Suitable for It

First, let’s talk about MiniMax. The official positioning of MiniMax-M2.5 is actually quite high. In press releases and open platform documentation, they push it towards scenarios like programming, tool calling, search, and office productivity, even emphasizing speed and cost advantages. I don’t completely disbelieve these claims, but I prefer to break them down. For me, what MiniMax is genuinely good at isn’t “the most complex development tasks,” but rather the following:

Data Cleaning

A lot of data cleaning is essentially manual labor involving semi-structured text.

Name unification
Field mapping
Anomaly labeling
Classification tagging
Table field completion What these types of tasks fear most is not the model being “dumb,” but rather inconsistent formatting or divergent outputs. As long as the model can reliably output results in JSON, tables, or fixed templates, it’s actually sufficient. While powerful models certainly can do this, using the most expensive tier of model just to clean fields is often not cost-effective.

Documentation Writing

Writing documentation is annoying, not difficult. When an interface changes, a process changes, or a field is modified, the documentation has to change accordingly. This process doesn’t actually require the model to have strong creativity; rather, it requires it not to over-exert itself and alter clearly defined things into something ambiguous. MiniMax is often more reliable for these kinds of tasks than one might expect. Especially when you have already prepared the context, it acts more like a capable documentation assistant rather than an actual engineer.

Solution Material Search

The official platform is also promoting search and tool calling, so this direction is fine. Many times, what we need is not for the model to “come up with an answer out of thin air,” but rather for it to first find relevant web pages, documents, announcements, or materials, and then organize them neatly. In this scenario, cheaper models like MiniMax are very valuable because searching, summarizing, and integrating are inherently high-frequency, mundane tasks. So my actual view is: MiniMax isn’t incapable; rather, it is better suited for the dirty, tiring, and repetitive tasks within a production pipeline. If you let it act as an assistant or general laborer, it is often competent; but if you ask it to handle the entire engineering process, the probability of disappointment increases.

Local 12B Models, Best Suited for Bringing Back These Tasks

Looking further down, the logic for local deployment is actually the same. When many people talk about local models, they inevitably ask one question: Can it replace the flagship cloud models? I think this question is flawed from the start. For local models around 12B, what has real practical value isn’t “proving that it can handle the most powerful tasks,” but rather bringing back those stable, repetitive, sensitive, low-profit, yet high-frequency tasks.

Translation

This is one of the most natural scenarios for local models. As explicitly mentioned in the official blog of Qwen2.5, it has enhanced capabilities for long-text generation, structured data understanding, and JSON output, and supports over 29 languages. This combination is inherently suitable for tasks like translation, bilingual rewriting, format standardization, and terminology normalization. Technical documentation, field descriptions, product introductions, and API comments—these items often have stable structures and fixed terminology. While local models might not produce the most elegant translations, they are usually sufficient.

Data Cleaning

This is also where local models are particularly realistic. Many spreadsheets, documents, and business materials that you might not want to upload to the cloud. Especially internal data, customer records, meeting minutes, and draft proposals—when privacy and permissions are involved, running it locally provides much more peace of mind. At this point, the significance of a local model around 12B isn’t “how smart it is,” but rather that “it’s on my machine, and it can reliably handle these dirty tasks.”

Fixed Format Rewriting

For example:

Meeting minutes organized into a fixed template
Product titles cleaned into a unified naming convention
Bug descriptions rewritten into ticket format
Mixed Chinese and English text cleaned into single-language versions

These types of tasks share consistent characteristics: clear rules, large batches, high repetition, low value per instance, but significant cumulative effort. This is exactly what local models are best suited for.

Can the 3060 12GB actually run a model around 12B?

I prefer to write about this realistically: “It can run it, but don’t get your hopes up too high.” Google provided a very useful VRAM table in the official documentation for Gemma 3. The Gemma 3 12B roughly requires:

About 20 GB of VRAM to load the full precision version.
About 12.2 GB to load the medium quantization version.
About 8.7 GB to load a lower VRAM consumption version. The official documentation also specifically reminds that this is only for model loading, and does not include prompt or runtime overhead. This sentence is very key. What does it mean? It means that running a model around 12B on a card like the 3060 12GB is not impossible, but the prerequisites are usually:
You are running a quantized version.
The context length should not be too long.
The task shouldn’t be too complex.
You accept average, or even slow, speed. If you are willing to accept these premises, then running a local 12B model is indeed feasible. Tasks like translation, summarization, table cleaning, and fixed format conversion are not exaggerated in this regard. Furthermore, the official repository for Qwen2.5-14B-Instruct-GGUF itself provides multiple quantization formats, which actually makes the intention very clear: models in this category are inherently adapted for the local inference ecosystem. So my conclusion has never been that “the 3060 12GB can easily handle a 12B model,” but rather: It can run these types of models, but it is better suited for work with low expectations, high repetition, and high privacy requirements.

Cheap Models and Local Models: It’s Not Just About Saving API Costs

When people talk about this, the first reaction is always saving money. Of course, saving money is important. But I think the greater value is that you start daring to outsource all those little tasks you used to avoid doing. Before, you might not have written a dedicated script just to clean up a few hundred data points. You also wouldn’t manually adjust dozens of pages of mixed Chinese and English documents to achieve uniform formatting. And you certainly wouldn’t read through every single webpage to gather materials for an ad-hoc proposal. Things are different now. As long as the cost is low enough and the barrier is low enough, these tasks that were previously considered “not worth the effort” suddenly become worthwhile. You no longer hesitate over whether or not to do it; instead, you just throw it to a cheap model or a local model to run through first. This is what I see as the most realistic change. Powerful models are responsible for tackling core problems, weaker models handle miscellaneous tasks, and local models provide fallback and batch processing. With this division of labor, the entire workflow becomes smooth.

Conclusion

So, the final word remains: don’t always try to make one model conquer everything. Models like MiniMax are weak in capability, but they aren’t useless. If you use them to tackle complex engineering tasks, vague requirements, or multi-turn reasoning, you will naturally be disappointed; however, if you use them for data cleaning, document drafting, or searching for proposal materials, they often work quite smoothly. The same applies to local models around 12B. Their purpose isn’t to prove that “I no longer need cloud flagships,” but rather to reliably move stable, repetitive, sensitive, and high-volume tasks back onto their own machines. Simply put: don’t let a weak model do what it is not good at. Place them in the right role, and they will have real value.

References

Writing Notes

Original Prompt

Minimax’s large model is weak in capability, but it’s fine for tasks like data cleaning, document writing, and searching for proposal materials; with the same logic, deploying a large model locally for translation or data cleaning work is also good. The model parameter count is around 12b, and even a local GPU like the RTX 3060 with 12GB can handle it.

Writing Outline Summary

Retained the core judgment of “don’t force weak models onto hard tasks,” and did not write it as a model leaderboard comparison.
The MiniMax section is mainly based on the official positioning for programming, searching, and office work, then applies this judgment back to real-world tasks like data cleaning, document handling, and information retrieval.
For local models, I selected two officially sourced options: Qwen2.5 and Gemma 3, one supporting multilingual and structured output, and the other supporting 12B size and VRAM usage.
The description for the 3060 12GB was intentionally phrased as “capable, but don’t get too carried away,” to avoid presenting quantized inference as an absolute conclusion.
In the conclusion, I re-categorized strong models, weak models, and local models based on their respective roles, making the main thread more focused.

The End of Low-Cost API Gateways: Large Model Experiences and the Impossible Triangle in March

Mon, 30 Mar 2026 20:00:00 +0800

Throughout March, I was constantly testing between various large model API hubs. It is indeed cheap. You can test out foreign models like ChatGPT, Claude, and Gemini for a small amount of money per month, which at first glance seems like finding an extremely cost-effective solution. However, after actually using it, I increasingly feel that this path has always been constrained by an impossible triangle: Quality, Stability, and Affordability—it is difficult for all three to be achieved simultaneously. By last weekend, the situation became quite clear. During the two days from 2026-03-28 to 2026-03-29, I felt a noticeable tightening of risk controls on ChatGPT channels, and Claude was no different. Many low-cost relays that were previously usable suddenly became unstable or even completely failed. For me, this basically signaled the temporary end of the low-cost API relay model.

What is an API Proxy/Gateway?

Let’s first clarify the concept. What is called an API Gateway (or “Proxy Station”) is not inherently a service provided by the model vendor itself, but rather an intermediary “forwarding layer” placed between the user and the upstream models. You send your request to the gateway, which then forwards it on your behalf to OpenAI, Anthropic, or other model providers, and finally returns the result to you. From the user’s perspective, it acts like a cheaper and more “flexible” unified entry point; from a technical and business model perspective, it is more like repackaging and redistributing upstream resources. The reason these services have been widely used is quite simple:

Low cost
Low barrier to entry
Wide variety of models available
For domestic users, it saves a lot of hassle related to registration, payment, and network environment setup. However, the problem lies precisely in this. Since it is not the official link, many conveniences are fundamentally built upon an “extra layer” rather than stable authorization.

How They Generally Work

The practices vary across different platforms, but the common patterns generally fall into a few categories.

1. Key Secondary Distribution

Some platforms essentially take their upstream API Keys and act as a unified forwarder, then divide the quota among downstream users. What you buy is their layer of package, not a package sold directly to you by OpenAI or Anthropic. The problem with this model is that if the upstream Key is restricted, the quota runs out, or the strategy is adjusted, the downstream experience will immediately fluctuate.

2. Account Pool Rotation

The term “account pool” often used in the industry can be understood as pooled management of a batch of account resources. The platform centralizes multiple accounts and calls them in rotation according to requests, which is used to distribute quota pressure and risk control pressure. The “account pool” here is not an official term from model vendors but rather a more colloquial or jargon-heavy description. It emphasizes the resource scheduling method rather than the product capability itself. Whoever has a larger pool appears more stable in the short term; however, this stability often vanishes instantly if upstream sources begin concentrated cleanups.

3. Reverse Engineering Wrappers

Another approach is not to use the official, standard APIs, but rather to study the request methods used by the web page or client, and then re-wrap these calling processes into an interface that “looks like an API” for users. This “reverse engineering” process, simply put, means not entering through the official door, but understanding how the system communicates by going through side entrances, windows, or even pipes, and then wrapping it in your own layer. The weakness of this method is also very obvious: what works today does not guarantee that it will work tomorrow. If the page structure, authentication methods, device verification, or behavioral policies change, the entire link may fail.

Actual Experience in March

After this intensive round of testing in March, my biggest takeaway wasn’t “cheap is surprisingly good,” but rather that you must continuously endure uncertainty. For the same requirement, one transit station might answer it well today, but tomorrow its intelligence might start declining; it might output stably in the morning but start throwing errors, timing out, or losing context in the evening. You think you are buying model capability, but what you are actually buying is a constantly fluctuating “probability service.” If you only use it for temporary testing of the model or for some light Q&A, this fluctuation is bearable. But once you put it into an actual workflow, the problems become very obvious.

Unstable quality, output level fluctuates wildly
Insufficient stability, prone to timeouts, errors, and disconnections
Poor context continuity, bad experience for long tasks
Model persona and style drift after switching upstream platforms
Spending time troubleshooting issues, the hidden cost is not low In other words, the biggest problem with low-cost transit services isn’t that they “aren’t cheap,” but that they transfer much of the certainty that should have been borne by official platforms back onto the user. On the surface, you save money, but in reality, you spend more time, mental energy, and reduce the predictability of your workflow.

Why I Say It Fell into the Impossible Triangle

After using it repeatedly recently, I increasingly feel that this type of relay service is destined to fall into an impossible triangle.

Quality

If you want to improve the quality, you must ensure that the upstream models are truly usable, have sufficient quotas, and experience as little throttling or degradation as possible. This task itself is not cheap.

Stability

If you want to achieve stability, you have to handle more risk control, account loss, network fluctuations, rate limiting, and backup lines. You even need to implement more complex scheduling and fallback mechanisms yourself. These are all costs.

Cost-Effective (or “Good Value”)

Once you have truly solidified the first two things, the price cannot continue to be suppressed so low. The ability to maintain ultra-low prices over the long term often indicates that the underlying cost has not been genuinely solved; it has merely been deferred or spread out across a future collective failure.

Therefore, many intermediaries that appear to be offering “high cost-performance” are actually maintaining a fragile balance. This balance holds up when things are calm, but once upstream risk controls tighten, this equilibrium can easily collapse.

Last weekend, it was pretty much obvious.

What really made me decide to give up on further tinkering was the change from 2026-03-28 to 2026-03-29. My own feeling is that the channels related to ChatGPT showed very noticeable tightening during those two days, and Claude’s risk control also strengthened in tandem. Previously, we could barely maintain a “usable” state by switching lines, models, or packages. That method suddenly no longer works. I don’t want to make an absolute judgment here like saying “the entire industry is dead,” because someone will always say they have some other usable channel. But from the perspective of actual utility for ordinary users, this path of low-cost API relay is at least no longer worth me continuing to invest time in. The concept of being cheap only has meaning when it’s predicated on “being able to complete the task reliably.” If even basic reliability cannot be guaranteed, low cost becomes an illusion.

Why This Pattern Was Inherently Fragile

On the surface, it might seem like the vendors are “deliberately restricting” people. However, if you look deeper, this type of pattern was built on a very fragile foundation to begin with.

Vendors like OpenAI and Anthropic did not design their products based on the logic of “secondary distribution gray resale.” Their official terms already have explicit restrictions regarding the reselling of API Keys, bypassing limitations, reverse engineering, or circumventing protective measures. OpenAI’s service agreement explicitly lists “buying, selling, or transferring API Keys,” “bypassing rate limits or protective measures,” and “circumventing usage restrictions” as prohibited activities; Anthropic’s commercial terms also clearly reserve space to constrain misuse, unauthorized access, and service abuse.

In other words, these relay stations are not innovating within an officially encouraged ecosystem; they are surviving in the gaps of official governance boundaries. As soon as model vendors start cleaning up seriously, this pattern will naturally be the first to take a hit.

There is also a very realistic background factor: many foreign model services already have barriers related to region, payment, and account systems. Since official support regions are limited, many users are blocked out, which created the demand for relays. But just because the demand exists doesn’t mean the pattern is stable. It only indicates that there is a market for gray alternative solutions; it does not guarantee long-term certainty.

Final Conclusion

After going through all this, my conclusion is actually simpler. Codex has a better cost-performance ratio, at least from my own usage experience; it seems more suitable for actual production environments. I haven’t heard much feedback about large-scale account bans in China, so the overall mental burden is lower. Of course, Codex isn’t without constraints. The five-hour limit and weekly limits are there, and when I use it now, I don’t act as recklessly as before—I don’t ask everything or dump everything on it to process. For many problems, I will first run them through my own mind, break them down myself, and then decide whether to spend the current quota. Looking at it this way, limits aren’t necessarily all bad. Objectively, they force the human brain to get re-involved—they force you to organize your thoughts first, make judgments first, and prioritize, rather than outsourcing all thinking. I have written about this before: a model that is slightly “less intelligent” isn’t necessarily all bad because it forces the user to maintain a basic level of critical thinking. Looking back now, this judgment still holds true. I am not considering Claude for now. It’s not that its capabilities are weak; quite the opposite, they are still very strong. But given the uncertainty of account bans even with personal paid subscriptions, it is not suitable as my primary solution at this stage. Regarding domestic models, I choose to continue observing. It’s not yet time for me to commit heavily or bind long-term. So, in the end, I returned to a very simple choice: subscribing to ChatGPT Plus and using it for now while continuing to watch how the large model industry evolves. Many times, the cheapest option is not necessarily the most cost-effective; the one that is the easiest to use is actually the true value proposition.

Reference Links

OpenAI Supported Regions: https://help.openai.com/en/articles/5347006-which-countries-and-territories-are-supported-by-openai
OpenAI Services Agreement: https://openai.com/policies/services-agreement/
Anthropic Commercial Terms: https://www.anthropic.com/legal/commercial-terms

Deep Dive: Memory Corruption and Cache Pollution in C++ with Static Lambdas

Mon, 16 Mar 2026 20:18:18 +0800

This article analyzes the bizarre phenomenon in C++ development where unordered_map::find returns an object with mismatched fields after a hit. The root cause lies in defining a static lambda within the function and using reference capture to capture local variables, leading to a dangling reference after the first call, triggering undefined behavior (UB) and polluting cache data in subsequent calls. It is recommended to address this issue by explicitly passing parameters instead of implicit capture, managing lifecycles properly, and utilizing Sanitizer tools.

When building high-performance market data services or distributed caches, developers often encounter a “ghost event”: after using a Key to explicitly find an object from std::unordered_map, when reading its internal fields, it turns out to be the data of another Key. This “identity misalignment” often points to an insidious C++ trap: conflict between static lambda and lifecycle capture resulting in undefined behavior (UB).

Failure Modes: Short Lifecycles of References “Long-Lastingly” Preserved

When optimizing performance, some developers are accustomed to defining static lambda functions within a function to reduce closure creation overhead. However, when combined with implicit capture using [&], this can lay a dangerous trap:

void UpdateCache(std::unordered_map<std::string, Tick>& cache, Tick& current_input) {
    // Dangerous: static extends the lambda's lifetime to process level
    // [&] captures a reference to the local variable current_input on the stack
    static auto patch_func = [&](auto it) {
        it->second.price = current_input.price; // The reference here is invalid at the second execution
    };

    auto it = cache.find(current_input.symbol);
    if (it != cache.end()) {
        patch_func(it); 
    }
}

Technical Principle Analysis

Lifecycle Misalignment: The static variable is initialized only once when the program reaches that line for the first time and persists throughout the entire process execution. This means the reference captured within patch_func to current_input is permanently fixed at the memory address in the stack frame at the time of its initial call.
Dangling Reference: After the first UpdateCache executes, the original stack frame is destroyed. When patch_func is called a second time, it continues to read and write to that now-invalidated address.
Hidden Memory Corruption: Due to the reuse of stack space, this address may contain new business data. Writing to it appears to be updating the value for the current Key, but in reality, it’s performing an illegal write operation on a “random” area of memory. This explains why find’s Key is A, and the read Value field is B.

Solutions and Engineering Practices

1. Eliminate Implicit Capture, Use Explicit Parameters

The most reliable approach is to keep the Lambda “stateless” (Stateless), passing all dependent objects through parameters.

// Recommended practice: Stateless lambda, explicitly pass src
auto patch = [](auto it, const Tick& src) {
    it->second.price = src.price;
};
patch(it, current_input);

2. Avoid using `static` to decorate closures

Inside a function, unless the closure does not capture any local variables, do not use static. Modern compilers are very good at optimizing non-static lambdas, and the performance boost from blindly using static is negligible, but it brings significant security risks.

3. Reinforcement Consistency Verification and Dynamic Detection

Assertion Checking: After updating the logic, add assert(it->first == it->second.symbol) to intercept “identity inconsistency” issues during development.
Tool Assistance: Enable ASan (AddressSanitizer) and UBSan (UndefinedBehaviorSanitizer) in the test environment. These tools can accurately capture behavior of accessing invalid stack memory and immediately report errors.

Conclusion

“Table lookup errors” in C++ are often just symptoms, the underlying truth is usually the failure of memory management contracts. Correct key lookup does not necessarily mean the value is still valid. When dealing with long-lived objects (such as static, global variables, and long-running callbacks), you must always be vigilant about whether the references they capture have been invalidated by stack frame destruction (“blown away”).

wrk vs. JMeter deep benchmarking

Fri, 19 Dec 2025 01:14:49 +0800

In internet system stress testing, we frequently encounter two tools with vastly different styles: one is extremely lightweight, pursuing extreme throughput—wrk; the other is feature-rich and simulates real business flows—JMeter.

Prompt: Outline the core ideas and write a科普 article (explanatory article): HTTP stress testing tools, wrk vs JMeter – what are the differences? What I know, wrk tends to use one thread with multiple connections for testing, while JMeter primarily employs a short connection mode, which can be adjusted via configuration to enable long polling.

Core Architecture: Multi-Threaded vs. Event-Driven

This is the fundamental reason for the performance gap between them.

1. JMeter: The “One Person, One Job” (Thread-per-Request) Model

JMeter is developed in Java and utilizes the classic multi-threaded model.

Logic: Each concurrent user (Virtual User) corresponds to a physical thread within the JVM.
Cost: Threads are an expensive resource. As concurrency increases to several thousand, context switching and memory consumption will significantly slow down the test machine itself, leading to the phenomenon of “the load generator collapsing first before it can actually crush the server.”

2. wrk: The Modern “Multi-Hand” System (Event-driven)

wrk is written in C and its core logic relies on Redis’s ae event loop framework (utilizing epoll/kqueue).

Logic: wrk doesn’t create a thread for each connection. It only starts a small number of threads (typically equal to the number of your CPU cores), and each thread manages thousands upon thousands of connections simultaneously through non-blocking I/O.
Advantages: This is what you referred to as “one thread, multiple connections.” It drastically reduces thread switching overhead, allowing single machines to achieve millions of RPS (Requests Per Second).

Connection Models: Skip Connections vs. Dense Connections

Regarding the connection patterns you mentioned, here’s a deeper level of detail:

1. JMeter’s “Heavy” and “Light”

JMeter defaults to simulating real user behavior.

Short Connection Bias: In default configurations, some older versions or specific configurations of JMeter may not actively reuse connections, leading to numerous TCP handshakes.
Tunability: You can enable long connection by checking the “KeepAlive” option in the HTTP Request or adjusting connection pool parameters in the user.properties file. However, even with long connections, limited by its thread model, it struggles to maintain tens of thousands of concurrent long connections.

2. wrk’s “Speed” and “Power”

wrk was designed with the intention of testing the performance of HTTP Keep-Alive.

Long Connection Strategy: wrk establishes a specified number of connections at the start of the test (-c parameter) and attempts to reuse these connections throughout the entire test.
Application Scenarios: It’s particularly well-suited for testing the throughput limits of Nginx, gateways (Gateways), or high-concurrency APIs under extreme long connection pressure.

Deep Comparison Table

Feature	wrk	Apache JMeter
Development Language	C/Lua (Scripting)	Java (GUI)

Deep Comparison Table

Feature	wrk	Apache JMeter
Concurrency Model	Event Driven (epoll/kqueue)	Multi-threaded (Thread-per-User)

Deep Comparison Table

Feature	wrk	Apache JMeter
Resource Consumption	Extremely Low, Huge Throughput on a Single Machine	Higher, Requires Distributed Cluster for High Concurrency

Deep Comparison Table

Feature	wrk	Apache JMeter
Business Complexity	Low, primarily for single URLs	High, supports multi-step scripts, assertions, and extractors

Deep Comparison Table

Feature	wrk	Apache JMeter
Test Scenarios	Static API Load Testing, Capacity Assessment	Complex Business Link Simulations, Functional Regression Testing

Deep Comparison Table

Feature	wrk	Apache JMeter
Reporting Capabilities	Only text summaries	Extremely rich, supports various charts and HTML reports

Summary: Which one should I choose?

These two tools are complementary rather than substitutional relationships:

Select Work

Want to test the maximum throughput (RPS) of the server.
The testing object is a single API or static resource.
Aim to push the largest traffic using the fewest test servers.
Familiar with Lua scripting to customize requests.

Choose JMeter

Requires simulating complex business processes (such as: Login -> Search Products -> Place Order -> Payment).
Needs a visual interface to observe response time distributions, error rates, and other detailed metrics.
Testing requires handling dynamic parameters (such as extracting a Token from the previous interface and passing it to the next interface).
The team is more accustomed to using graphical tools rather than command-line interfaces.

Detailed Explanation of How Passkeys Work and Their Future

Thu, 04 Dec 2025 22:14:06 +0800

Background introduction: Tencent’s CNB platform only supports WeChat login, with no conventional email account method, leading to several guys in the group complaining about it daily – it’s getting annoying. The Tencent product manager came up with a compromise solution: support Passkey login.

Every day, we are repeating a dangerous action: entering passwords. Despite our complex rules (uppercase, special symbols, numbers), data breaches, phishing attacks, and the frustration of “forgotten password” issues continue to trouble everyone.

Tech giants (Apple, Google, Microsoft) along with the FIDO Alliance have provided the ultimate solution: Passkey (Passcode Key). It’s not just a “replacement” for passwords; it completely “eliminates” them.

The login process has shifted from verifying passwords to verifying the current device’s trustworthiness.

Explain how Passkey works, how it can be used for login management, collect relevant content, and output an article for internet publication

What is a Passkey?

Simply put, a Passkey is a digital credential stored on your device. It replaces the traditional “username + password.” When you log into websites that support Passkeys (such as Google, GitHub, Adobe), you don’t need to enter any characters – you just verify using facial recognition (Face ID), fingerprint (Touch ID), or device PIN, and you’re instantly logged in.

Key Difference: A password is a string of characters you remember (easily stolen, guessed, or forgotten); a Passkey is an asset you own – an encrypted key stored within the device hardware.

How Passkeys Work: Asymmetric Encryption

Passkey is built on the WebAuthn standard and the FIDO2 protocol. To understand it, we need to understand Public Key Cryptography.

Imagine a passkey as a pair of “keys” and “locks”:

Private Key:
- Where is it stored? Securely stored on your device (such as iPhone’s secure enclave, computer’s TPM module, or password manager).
- Characteristics: Extremely sensitive, never sent to the server, and doesn’t leave your device.
- Function: It’s your “digital signature pen.”
Public Key:
- Where is it stored? Uploaded and stored on the website/app’s server.
- Characteristics: Publicly visible, not sensitive.
- Function: It’s used to verify your signature – like a “money checker.”

Handshake Details (The Process)

When you log in using a Passkey, a sophisticated “challenge-response” mechanism takes place:

Initiate Login: You click “Login,” and the website server sends a random mathematical problem (Challenge) to your device.
Local Verification: Your phone/computer displays a prompt requesting you unlock it using biometrics (face recognition/fingerprint).
- Note: This step is solely for authorizing the device to use its private key; biometric data itself is not uploaded.
Digital Signature: Upon successful unlocking, the device uses its private key to “sign” the mathematical problem and sends the signature result back to the server.
Server Verification: The server uses the public key you previously provided to verify this signature. If it’s valid, the server confirms that “the person who has validated the key indeed holds the private key,” thereby allowing login.

Why Passkeys Are Significantly More Secure Than Passwords?

Passkeys address the three major pain points of traditional passwords:

Completely Immune to “Phishing Attacks” (Anti-Phishing)

This is the most robust feature of Passkey. The Passkey protocol enforces the inclusion of Origin Binding.

Scenario: A hacker creates a fake g00gle.com to trick you into logging in.
Result: Your browser and system will detect that the current domain does not match the domain registered for your Passkey, google.com, and refuse to initiate authentication. You don’t even have a chance to “fall for it” and enter your password.

Server Leak Was Useless

Even if hackers breached Google’s servers and stole all the databases, they would only have obtained the public key.

The public key cannot be used to derive the private key. Holding the public key is like holding a lock; but without the key (the private key which resides on your phone), the hacker couldn’t access your account.

No “Weak Passwords”

Users no longer need to set weak passwords like “123456,” as the keys are strong, encrypted data generated by algorithms.

Previously, we relied on tools like 1Password, LastPass, or Chrome browser to store lengthy strings of characters. Now, login management is undergoing a fundamental shift.

Cross-Device Sync (Passkey Sync)

Early hardware keys (such as YubiKeys) are easily lost. Current Passkeys support cloud synchronization:

Apple Ecosystem: Through iCloud Keychain sync. Passkeys you create on your iPhone are automatically available on your Mac.
Google Ecosystem: Through Google Password Manager sync for Android and Chrome.
Third-Party Management: Tools like 1Password, Dashlane, etc., fully support Passkey. This means you can seamlessly sign in across ecosystems (using a Passkey stored in your iPhone on a Windows computer).

What do you do if you want to log in to a PC at an internet cafe (Windows) but your Passkey is on your iPhone?

Select “Login via Another Device” on the web page.
A QR code (FIDO Cross-Device Flow) appears on the screen.
Use your iPhone’s camera to scan the QR code.
Your phone establishes a Near Field Connection with the computer via Bluetooth (to verify you are present) and performs biometric authentication.
Login is successful on the PC.

From “Managing Secrets” to “Managing Trust”

Future login management will no longer be about checking individual plaintext passwords, but rather managing trusted devices.

You can see: “My GitHub account is bound to my iPhone and MacBook.”
If your phone is lost or stolen, you simply revoke the device’s public key access rights from the server (or cloud account).

Table Comparison: Password vs. Passkey

Dimension	Traditional Password	Passkey
Cognitive Burden	High (Requires memorizing complex characters)	None (No need to memorize)

Table Comparison: Passwords vs. Passkeys

Dimension	Traditional Passwords	Passkeys
Phishing Risk	High (Susceptible to being tricked into entering)	Zero (Domain Binding)

Table Comparison: Passwords vs. Passkeys

Dimension	Traditional Passwords	Passkeys
Server Leakage	Dangerous (Requires Brute-Force / Password Reset)	Secure (Key Leakage Has No Impact)

Table Comparison: Password vs. Passkey

Dimension	Traditional Password	Passkey
Login Experience	Slow (typing or copy-pasting)	Fast (one-tap biometric verification)

Table Comparison: Password vs. Passkey

Dimension	Traditional Password	Passkey
Reliance	Relies on memory or a password manager	Relies on device (phone/computer)

Current Challenges and Future

Despite the great potential of Passkeys, widespread adoption still requires time:

Platform Barriers: While standards are unified, Apple, Google, and Microsoft offer the smoothest experiences within their respective ecosystems. Cross-ecosystem transitions (such as an Android phone paired with an iPad) are feasible but still involve some friction.
Device Dependency: Losing all trusted devices without a cloud backup can make account recovery difficult (typically requiring a backup recovery code).
Legacy System Compatibility: Many older websites and internal corporate networks do not yet support the WebAuthn standard.

Summary

Passkey isn’t an upgrade to passwords; it’s a fundamental reconstruction of internet identity verification. It leverages modern device biometric capabilities and public-key cryptography, elevating security to a financial level while simplifying the user experience to the extreme.

For average users, enabling Passkeys on supported platforms (Google, Apple, Microsoft, Amazon, etc.) is the highest return on investment for improving personal digital security.

Is AI Getting “Dumber”? Uncovering the Game Between Parameter Accuracy and Inference Costs

Thu, 04 Dec 2025 21:58:09 +0800

Recently mingling in various programming large model communication circles, the most complained about thing is model degradation.

Models deployed on local desktop computers are quantized models, essentially downgraded versions.
With “vibe coding” so popular, could it be that the content output by current large models is the most valuable product – code?

This round of prompts received one optimization, which coincided with model degradation, and the large model’s solution was to optimize the prompt, with more detailed task planning and clearer output requirements.

Addressing this issue, provide suitable prompts: Many large factories have now provided large model services, and users sometimes feedback that models are degrading; from a professional perspective, considering parameter precision and inference cost, write an article;科普文 (科普 article), don’t be too long. Optimized version:

You are a senior AI industry technical column writer. Your goal is to write a professional but accessible Chinese科普 article for general readers.

Please step by step think and write:
1. Outline drafting: First, please draft a clear three-paragraph article structure outline (e.g., Introduction, Precision Analysis, Cost & Architecture Analysis, Conclusion).
2. Content focus: The article must deeply and clearly explain the phenomenon of “model degradation” reported by users, and the core analysis points must revolve around **parameter precision (quantization)** and **inference operational cost**.
3. Word count and format: The final article length must be controlled at 600 words or less. Please use clear `##` headings to divide paragraphs, and use **bold** to emphasize key professional terms (such as: quantization, MoE, FP16).

Introduction

To ensure clarity and align with a科普 (explanatory) approach, I’ve established the following structure:

Introduction: Confirming user perception of reality and introducing the core conflict – in the context of rapidly growing user scales, service providers must balance “intellectual peak” with “service availability.”
Precision Analysis (Core Point 1): Explaining the relationship between parameters and precision. By comparing FP16 with quantization (INT8/INT4), we use the analogy of “image compression” to illustrate why a drop in precision leads to degraded logic.
Cost & Architecture (Core Point 2): Analyzing inference cost. We introduce the MoE architecture and sparse activation mechanisms, explaining how models “slack off” to save money and accelerate performance.
Conclusion: Summarizing that this “degradation of intelligence” is an inevitable path for commercialization and widespread adoption (the impossibility triangle).

Article Body

Recently, many users have reported that large models seem less “intelligent” than before: logic has become simpler, instruction following has deteriorated, and even empty talk (“car轱辘话”) has emerged. This isn’t entirely an illusion. From a technical perspective, this “decline in intelligence” phenomenon is often a technological compromise made by major companies when facing massive user bases, balancing inference costs with service speed.

Precision “Shrinking”: The Cost of 4K to 720P

The “intelligence” of large models is primarily stored in trillions of parameters. Ideally, these parameters run in high-precision FP16 (16-bit floating-point) format, capturing extremely subtle semantic differences. However, this high precision means significant memory usage and slow computation speeds.

To allow hundreds of millions of users to use the service smoothly, service providers generally adopt quantization technology. This is a technique that compresses parameter precision from FP16 to INT8 or even INT4.

This is like compressing a 4K high-definition movie into a 720P stream: although the plot (overall logic) hasn’t changed, the visual details (small logical associations, complex instruction execution details) are lost. This “lossy compression” directly leads to a decline in model performance when handling complex tasks, giving users the sensation of “getting dumber.”

Cost Pressure: Letting the Brain “Partially Rest”

Besides accuracy, the operational cost of inference is another key factor. Every time you ask an AI a question, servers perform massive matrix calculations, leading to exorbitant electricity bills and hardware wear and tear.

To reduce costs, modern large models often adopt MoE (Mixture of Experts) architecture. Unlike traditional models that activate all neurons every time, MoE employs a sparse activation strategy: for your question, the system only activates a small subset of relevant “expert” networks, while the rest remain dormant.

Although this significantly reduces computational load, if the routing algorithm (Router) distributes your complex mathematical problems to “literary experts” for load balancing or power saving, or sacrifices the number of expert calls in favor of speed, the output quality will inevitably fluctuate and decline.

Conclusion

Therefore, what is being referred to as “diminishment” – in essence, is the inevitable pain associated with AI’s transition from the laboratory to large-scale commercialization. The restraint on parameter precision and the efficient design of model architecture are what manufacturers seek to balance between “absolute intelligence” and “cost efficiency,” allowing everyone to affordably use AI.

Cloudflare experienced a global network outage, causing websites like X (formerly Twitter) and ChatGPT to crash. The stock price suffered a setback ahead of the market open.

Tue, 18 Nov 2025 22:34:02 +0800

This site’s main domain is hosted on GitHub Pages, and the blog’s backup subdomain is on Vercel with acceleration. It has been deployed through Cloudflare. The backend management page looks too outdated and unappealing, which is a major deterrent. Initially, it was a quiet, unassuming piece of content; however, constant messages in the blogging circles disrupted this tranquility – opening it up, CF went down, and many friends’ websites couldn’t be accessed.

As an internet infrastructure component, such prolonged downtime is unacceptable. The stock price didn’t plummet dramatically, which I hadn’t anticipated.

Cloudflare outage; many well-known websites are inaccessible. Pre-event stock drop, compiling relevant content, outputting articles

November 18, 2025 (Tuesday) – This morning (November 18th), Cloudflare, a major global internet infrastructure services provider, experienced a widespread network outage before the opening of U.S. markets, causing thousands of well-known websites and applications that rely on its services to become paralyzed or difficult to access worldwide.

As a result of this incident, Cloudflare’s (NYSE: NET) stock price fell sharply during pre-market trading.

🌍 Outage Affects Global Services as Popular Platforms “Go Down”

The outage began on Tuesday morning (approximately 7:00 AM Eastern Time, 11:48 GMT) and was reported globally. Users across the world were unable to access a number of popular services including social media platform X (formerly Twitter), artificial intelligence service OpenAI (ChatGPT), music streaming service Spotify, game platforms League of Legends and Valorant, as well as numerous cryptocurrency exchanges (such as BitMEX, DefiLlama), among others. Users attempting to access these websites generally encountered “500 Internal Server Error” messages. Given Cloudflare’s role as an “intermediary” in the internet, providing content delivery (CDN) and security protection (DDoS) for millions of websites worldwide, any “jitter” within its network immediately triggered a global chain reaction.

📉 Market Reaction Was Rapid, (NET) Pre-Market Stock Drop

As a key infrastructure component for the internet, Cloudflare’s service stability is of significant interest to investors. Following the release of the outage news, the capital markets reacted quickly. Cloudflare’s (NYSE: NET) stock experienced heavy selling in pre-market trading on Tuesday. According to financial data, its price fell approximately 3.5% to 4%, dropping from a previous day’s close of around $202.25 on November 17th to levels as low as $195.

This reflects investors’ concerns regarding the severity of the service interruption and its potential impact on the company’s reputation and short-term revenue.

Official Response: Issue Identified and Under Repair

Cloudflare’s official team quickly confirmed this incident on its System Status page. The company updated the status at 13:09 UTC (November 18th) this afternoon (Beijing time) stating:

“The issue has been identified and remediation is being implemented.”

🛠️ Official Response: Issue Identified and Under Repair

According to Cloudflare’s report, this incident has been classified as a “global network experiencing issues” (global network encountering problems) and accompanied by “widespread 500 errors.”

Cloudflare stated that they are working to mitigate the issue and have observed service beginning to recover, but cautioned that customers may continue to observe “above normal error rates” during ongoing repair work.

As of this writing, access to some websites has begun to gradually restore, but global network latency and error rates remain high. We will continue to monitor the situation’s further development.

Understanding MySQL Gaps Locks: From Principles to Enterprise-Grade Selection

Tue, 18 Nov 2025 22:29:25 +0800

System data synchronization between systems, recently when changes occurred, the data volume of a certain interface increased, frequent deadlocks happened. After colleagues investigated, they found that it was caused by gap locks. MySQL is not used much, so I’ll record it down.

mysql gap locks, background, principle, how to disable gap locks, what problems will occur if disabled; you are a senior database DBA, provide an extended analysis and supplement for my problem. What is phantom read? Which RC or RR mode do mainstream large factories choose? I am in the financial industry, securities system, which scheme do you recommend me to use? Compile the above questions and output documentation.

In MySQL, especially in high-concurrency scenarios, “locks” are a topic that cannot be avoided. Gap Locks are a core feature of the InnoDB engine, but they are also often the root cause of performance bottlenecks and deadlocks.

This article will help you quickly understand the core, pros and cons of gap locks, and provide you (especially those in the financial industry) with clear architectural selection recommendations.

What are Phantom Reads and Gap Locks?

Phantom Read: In a transaction, you initially read a range of data (e.g., ID > 100) returning 10 records. After another transaction inserts a record with ID=101 and commits, when you re-read the same range, it now returns 11 records. This “extra” new record is like an illusion, hence the name “Phantom Read.”
Gap Lock: MySQL introduces the Gap Lock in the RR (Repeatable Read) isolation level to address the issue of “Phantom Reads.” It doesn’t lock individual data rows; instead, it locks the “gaps” between data.
- Purpose: To prevent other transactions from inserting new data into these “gaps.”
- Cost: The scope of the lock expands, reducing concurrency and increasing the risk of deadlocks.

How to “Close” Gap Locks?

The most direct method is to lower the database isolation level from RR (Repeatable Read) to RC (Read Committed).

-- Set the current session isolation level to RC
SET SESSION transaction_isolation = 'READ-COMMITTED';

-- Permanent modification (requires modifying my.cnf and restarting)
-- [mysqld]
-- transaction-isolation = READ-COMMITTED

⚠️ Consequences of Closure (i.e., RC-level Features)

Performance Improvement: Without gap locks, the granularity of locks is smaller, resulting in a significant increase in concurrency throughput.
Reduced Deadlocks: Most deadlocks caused by gap locks will disappear.
Phantom Reads/Non-Repeatable Reads: This is a “feature” of RC-level features – results of two queries within a transaction may be inconsistent.
[Core Requirement] Must Be Used with ROW Format Binlog:

If using RC-level, it’s mandatory to set binlog_format to ROW. Otherwise, due to the lack of gap lock protection during master-slave replication, data inconsistency will occur between the master and slave.

Mainstream Factory Choices: RC or RR?

Answer: The vast majority of leading internet companies choose RC (Read Committed) + ROW Binlog mode.

Reason: Internet businesses (such as e-commerce and social media) pursue extreme high concurrency. They cannot tolerate the frequent lock waiting and deadlocks caused by RR gap locks.
Trade-off: They choose to abandon database-level “repeatable read,” instead relying on optimistic locking (such as version numbers, CAS) at the application layer to ensure the logical consistency of key business areas (such as inventory and balance).

How Should the Financial Industry (Brokerage Firms) Choose?

For brokerage systems with extremely high data consistency requirements, it is recommended to implement scenario-based governance:

Option 1: Core Trading System (High Concurrency, Low Latency)

Recommendation: RC (Read Committed) + ROW Binlog + Application-Level Optimistic Locking

Scenario: Order Matching, Placing Orders, Funds Deduction.
Reasoning: The concurrency pressure of trading systems is comparable to that of flash sales on the internet. RR’s gap locking will become a performance bottleneck, leading to severe lock competition and even deadlocks – something a trading system cannot tolerate.
Countermeasures: Use RC to guarantee high performance while ensuring financial security within application code (e.g., in Java) through UPDATE ... WHERE balance > ? or using CAS version number mechanisms to prevent overselling.

Option Two: Clearing and Reconciliation System (Batch Processing, High Consistency)

Recommendation: RR (Repeatable Read)

Scenario: Post-settlement bulk reconciliation, report generation, daily final settlement.
Reasoning: Batch processing tasks require a “consistent snapshot” to run on. It needs the capabilities provided by RR level, ensuring that data does not experience “phantom reads” throughout the entire statistical process, guaranteeing the general ledger is accurate.

Summary

Isolation Level	Advantages	Disadvantages	Suitable Scenarios
RR (Default)	Solves phantom reads, high data consistency	Low concurrency, prone to deadlocks (gap locks)	Reporting, clearing, scenarios with extremely high data consistency requirements and low concurrency

Summary

Isolation Level	Advantages	Disadvantages	Use Cases
RC	High Concurrency, Fewer Deadlocks	Phantom Reads, Non-Repeatable Reads	High Concurrency Core Business (e.g., E-commerce, Financial Transactions), Combined with ROW Binlog

Summary

For securities broker core systems, RC + ROW Binlog is the mainstream architecture solution that balances performance and consistency; however, it requires the development team to take on more responsibility for ensuring data consistency at the application layer.

Local Development Pain: Why Can't You Delete `nul` Files? A Solution to the “Composite” File System Problem

Sat, 08 Nov 2025 16:37:46 +0800

In the daily work of software development, we often encounter tricky “minor issues” that seem simple but can consume us hours of valuable time. One such “disaster zone” is deleting specific files – particularly those generated unexpectedly by toolchains.

I encountered a “level-boss” problem: during local development, a file named nul inexplicably appeared in my project. I tried Windows Explorer, the CMD command line, but the system displayed “File not found” or “Cannot delete.” This file was like a ghost, stubbornly residing in my project directory.

Phase One: Routine Attempts and the Ineffective Solution of “Standard” Approaches

When I encounter this problem, my first instinct is to create a nul file. Why is nul special?

Developers familiar with Windows history may know that nul is a “trap.” In Windows (and earlier DOS) systems, NUL, CON, PRN, AUX, etc., are reserved device names. NUL represents the “null device” (similar to Unix/Linux’s /dev/null).

When the Windows file system API sees you trying to operate a “file” named nul, it treats it as an operation on this “null device,” rather than a filename with the same name. Therefore, all standard file operations (such as deleting or renaming) fail. How are nul files created?

This is usually the fault of cross-platform development tools (like Git, Node.js scripts, Python scripts, etc.). These tools may be based on POSIX (Unix-like) standards, in which case nul is simply a regular filename to them. When they run on Windows, they sometimes bypass standard APIs and create this “Windows-unfriendly” file. “Standard” Solutions Recommended Online

I quickly searched the web and found that I wasn’t the first person to encounter this problem. The community offered several “advanced” solutions that were widely accepted:

Use the \\.\ syntax: In CMD, use the special “long path” syntax to bypass Windows name checking.
```
del \\.\C:\your\project\path\nul
```
Use Git Bash: Git Bash provides a lightweight Unix environment, which doesn’t treat nul as a special device.
```
rm nul
```
Use WSL (Windows Subsystem for Linux): Enter WSL, mount the Windows disk, and use the Linux rm command to delete.
```
rm /mnt/c/your/project/path/nul
```

However, none of these methods worked for me! Whether it was WSL or Git Bash, when I tried rm nul, the system reported a “No such file or directory” error. This led me to ponder the problem seemed more complex than I initially thought.

Stage Two: A Flash of Insight – Is it “Multiple Problem Overlapping”?

If the nul file actually existed, why were even Unix tools saying “cannot find” it?

I began to suspect that the problem wasn’t just with the nul file itself, but also with its “habitat” – the directory where it was located?

I immediately opened Git Bash (this was key, because Windows Explorer might not display anomalies), navigated to the directory containing the nul file, and then executed ls -la (list all files, including hidden ones, and show detailed information).

That’s when I finally discovered the “blind spot”: the directory where the nul file was stored actually contained an illegal character within its name!

Stage Two: A Flash of Insight – Is it “Multiple Problem Overlap”?

In my case, this directory name might be one ending with a space or a period (.) or containing special characters (like ?, *, :) that development tools “carry along” when syncing across platforms—these are all equally problematic.

For example, a directory displayed in Git Bash as "my-app " (note the trailing space) or "my-app.".

That’s where the problem lies!

Problem A: I have an “illegal” file named nul.
Problem B: I have an “illegal” directory named "my-app ".

When I try rm /path/to/"my-app "/nul, both the Windows system and Unix tools are “confused.” The Windows API cannot correctly parse this path containing illegal characters; while Git Bash or WSL might “see” this illegal directory, it may fail to access its internal nul file due to complex path parsing issues.

Phase Three: Cutting Off the Root – Targeting the Path, Eliminating Everything in One Go

Now that we’ve confirmed it was a problem with both “file path” and “filename,” the solution became clear: don’t try to delete that nul file; instead, directly delete that “illegitimate” parent directory!

My final resolution steps were as follows:

Open Git Bash: This is the only tool capable of correctly “seeing” and handling these illegal names.
Navigate to the parent directory of the “problem directory”:
Verify the true name of the “problem directory”:
Execute the “Ultimate Delete”: Using the rm command with the -r (recursive) and -f (force) options, combined with quotes, to delete the entire directory.

After executing the command, the troublesome directory containing the nul file, which itself had an invalid name, was finally completely removed from my filesystem.

Keepalived + HAProxy for High Availability Load Balancing

Fri, 19 Sep 2025 09:45:55 +0800

In modern internet architectures, high availability is a crucial consideration in system design. This article will detail how to use Keepalived and HAProxy to build a highly available load balancing cluster, ensuring service continuity and reliability.

The practical configuration section was not validated, and the article planning relies on AI completion

(Since I cannot see the image, I’m simply repeating the markdown as it was provided.)

Technical Overview

Keepalived Introduction

Keepalived is a high availability solution based on the VRRP (Virtual Router Redundancy Protocol) protocol, primarily used to implement server failover and load balancing.

Key Features:

VRRP Protocol Support: Enables virtual IP address master/slave switching
Health Checks: Monitors service status and automatically performs failover
Simple Configuration: Complex high availability architectures can be achieved through configuration files alone
Lightweight: Low resource consumption and excellent performance

Working Principle: Keepalived utilizes the VRRP protocol to share a virtual IP address across multiple servers. In normal operation, the master server holds the virtual IP and provides services; when the master server fails, the backup server automatically takes over the virtual IP, ensuring service continuity.

HAProxy Overview

HAProxy is a high-performance load balancer and reverse proxy server, widely used in high concurrency scenarios.

Key Features:

Load Balancing: Supports various load balancing algorithms
Health Checks: Monitors backend server status in real-time
SSL Termination: Supports HTTPS traffic handling
Statistical Monitoring: Provides detailed running state statistics

Application Scenarios:

Web service load balancing
Database connection pooling
Microservice gateways
API interface proxy

Architecture Design

Overall Architecture

                    ┌─────────────────┐
                    │   Client        │
                    └─────────┬───────┘
                              │
                    ┌─────────▼───────┐
                    │  Virtual IP     │
                    │  (VIP)          │
                    └─────────┬───────┘
                              │
              ┌───────────────┼───────────────┐
              │               │               │
    ┌─────────▼───────┐              ┌─────────▼───────┐
    │   HAProxy-1     │              │   HAProxy-2     │
    │  (Master)       │◄────────────►│   (Backup)      │
    │  + Keepalived   │   VRRP       │  + Keepalived   │
    └─────────┬───────┘              └─────────┬───────┘
              │                                │
              └──────────┬─────────────────────┘
                         │
        ┌────────────────┼────────────────┐
        │                │                │
┌───────▼───────┐ ┌──────▼──────┐ ┌───────▼───────┐
│  Web Server 1 │ │ Web Server 2│ │  Web Server 3 │
│   Backend     │ │   Backend   │ │   Backend     │
└───────────────┘ └─────────────┘ └───────────────┘

Component Description

Virtual IP (VIP): A unified entry point for clients to access.
HAProxy Master-Backup Nodes: Provides load balancing services and achieves high availability through Keepalived.
Backend Servers: The actual web servers providing the service.

Environment Setup

Server Planning

Role	IP Address	Hostname	Service
HAProxy Master Node	192.168.1.10	lb-master	HAProxy + Keepalived

Server Planning

Role	IP Address	Hostname	Service
HAProxy Backup Node	192.168.1.11	lb-backup	HAProxy + Keepalived

Server Planning

Role	IP Address	Hostname	Service
Virtual IP	192.168.1.100	-	VIP

Server Planning

Role	IP Address	Hostname	Service

Server Planning

Role	IP Address	Hostname	Service

Server Planning

Role	IP Address	Hostname	Service

Software Installation

Install the necessary software on the HAProxy master and backup nodes:

# CentOS/RHEL
yum install -y haproxy keepalived

# Ubuntu/Debian
apt-get update
apt-get install -y haproxy keepalived

# Enable services to start automatically on boot
systemctl enable haproxy keepalived

Keepalived Configuration

Master Node Configuration (lb-master)

Create the configuration file /etc/keepalived/keepalived.conf:

! Configuration File for keepalived

global_defs {
    router_id LB_MASTER
    script_user root
    enable_script_security
}

# Script to check HAProxy service status
vrrp_script chk_haproxy {
    script "/etc/keepalived/check_haproxy.sh"
    interval 2
    weight -2
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass mypassword123
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        chk_haproxy
    }

    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

LB Backup Configuration

Create the configuration file /etc/keepalived/keepalived.conf:

! Configuration File for keepalived

global_defs {
    router_id LB_BACKUP
    script_user root
    enable_script_security
}

vrrp_script chk_haproxy {
    script "/etc/keepalived/check_haproxy.sh"
    interval 2
    weight -2
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass mypassword123
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        chk_haproxy
    }

    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

Health Check Script

Create the HAProxy health check script /etc/keepalived/check_haproxy.sh:

#!/bin/bash

# Check if the haproxy process is running
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
    # Attempt to start HAProxy
    systemctl start haproxy
    sleep 2

    # Check again, if it's still not running exit
    if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
        exit 1
    fi
fi

# Check if HAProxy port is listening
if ! netstat -tuln | grep -q ":80 "; then
    exit 1
fi

exit 0

State Notification Script

Create the state notification script /etc/keepalived/notify.sh:

#!/bin/bash

TYPE=$1
NAME=$2
STATE=$3

case $STATE in
    "MASTER")
        echo "$(date): Became MASTER" >> /var/log/keepalived-state.log
        ;;
    "BACKUP")
        echo "$(date): Became BACKUP" >> /var/log/keepalived-state.log
        ;;
    "FAULT")
        echo "$(date): Fault detected" >> /var/log/keepalived-state.log
        ;;
    *)
        echo "$(date): Unknown state: $STATE" >> /var/log/keepalived-state.log
        ;;
esac

Set script execution permissions:

chmod +x /etc/keepalived/check_haproxy.sh
chmod +x /etc/keepalived/notify.sh

HAProxy Configuration

Main Configuration

Create the same HAProxy configuration file /etc/haproxy/haproxy.cfg on the master node:

global
    log 127.0.0.1:514 local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

defaults
    mode http
    log global
    option httplog
    option dontlognull
    option log-health-checks
    option forwardfor except 127.0.0.0/8
    option redispatch
    retries 3
    timeout http-request 10s
    timeout queue 1m
    timeout connect 10s
    timeout client 1m
    timeout server 1m
    timeout http-keep-alive 10s
    timeout check 10s
    maxconn 3000

# Statistics page configuration
listen stats
    bind *:8080
    stats enable
    stats uri /stats
    stats realm HAProxy\ Statistics
    stats auth admin:password123
    stats refresh 30s

# Frontend configuration
frontend web_frontend
    bind *:80
    default_backend web_servers

# Backend server configuration
backend web_servers
    balance roundrobin
    option httpchk GET /health
    
    server web1 192.168.1.20:80 check inter 2000 rise 2 fall 3
    server web2 192.168.1.21:80 check inter 2000 rise 2 fall 3
    server web3 192.168.1.22:80 check inter 2000 rise 2 fall 3

Configuration Instructions

Global Configuration:

log: Log configuration
chroot: Security sandbox
stats socket: Management interface
daemon: Background execution

Default Configuration:

mode http: HTTP mode
balance roundrobin: Round robin load balancing
option httpchk: HTTP health check
timeout: Various timeout settings

Backend Servers:

check: Enable health checks
inter 2000: Check interval of 2 seconds
rise 2: Mark as available after 2 consecutive successful checks
fall 3: Mark as unavailable after 3 consecutive failed checks

Service Startup and Testing

Start Service

Start the service on the master and backup nodes:

# Start HAProxy
systemctl start haproxy
systemctl status haproxy

# Start Keepalived
systemctl start keepalived
systemctl status keepalived

Verify VIP Binding

Check if the virtual IP is correctly bound:

# View IP address on the master node
ip addr show

# You should see output similar to:
# eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
#     inet 192.168.1.10/24 brd 192.168.1.255 scope global eth0
#     inet 192.168.1.100/24 scope global secondary eth0:0

Functional Testing

1. Load Balancing Testing

# Repeatedly access the VIP and observe request distribution
for i in {1..10}; do
    curl -s http://192.168.1.100/ | grep "Server"
done

2. Failover Testing

# Stop the HAProxy service on the primary node
systemctl stop haproxy

# Observe if the VIP switches to the backup node
ip addr show

# Test if the service is working normally
curl http://192.168.1.100/

3. Backend Server Failure Testing

# Stop one of the web servers
# On the web1 server:
systemctl stop nginx

# Observe the HAProxy statistics page
curl http://192.168.1.100:8080/stats

Monitoring and Maintenance

Log Monitoring

HAProxy Logs

# View HAProxy logs
tail -f /var/log/haproxy.log

# View access statistics
grep "HTTP/1.1" /var/log/haproxy.log | tail -20

Keepalived Logs

# View Keepalived logs
tail -f /var/log/messages | grep keepalived

# View state change logs
tail -f /var/log/keepalived-state.log

Performance Monitoring

Statistical Page Monitoring

Access the HAProxy statistics page: http://192.168.1.100:8080/stats Key Metrics:

Session Rate: Session rate
Session Total: Total number of sessions
Bytes In/Out: Traffic statistics
Response Time: Response time
Server Status: Server status

Command Line Monitoring

# Check HAProxy process status
ps aux | grep haproxy

# Check port listening status
netstat -tuln | grep -E "(80|8080)"

# Check connection count
ss -ant | grep :80 | wc -l

Troubleshooting FAQs

1. VIP cannot be switched

Problem Description: After the master node fails, the VIP does not switch to the backup node. Troubleshooting Steps:

# Check Keepalived configuration
keepalived -t -f /etc/keepalived/keepalived.conf

# View VRRP communication
tcpdump -i eth0 vrrp

# Check firewall settings
iptables -L | grep vrrp

Solution:

Ensure that VRRP protocol communication is normal.
Check network interface configuration.
Verify authentication password consistency.

2. Health Check Failed

Problem Description: Backend server marked as unavailable Troubleshooting Steps:

# Manually execute health check
curl -I http://192.168.1.20/health

# View HAProxy logs
grep "Health check" /var/log/haproxy.log

Solution:

Ensure the health check URL is accessible
Adjust the check interval and thresholds
Check the backend server status

3. Load Unbalance

Problem Description: Requests are not evenly distributed to the backend servers. Troubleshooting Steps:

# View statistics page
curl -s http://192.168.1.100:8080/stats

# Analyze access logs
awk '{print $6}' /var/log/haproxy.log | sort | uniq -c

Solution:

Check the load balancing algorithm configuration
Verify server weights settings
Consider session persistence requirements

Optimization Suggestions

1. Performance Optimization

# Adjust system parameters
echo 'net.core.somaxconn = 65535' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_max_syn_backlog = 65535' >> /etc/sysctl.conf
sysctl -p

# Optimize HAProxy configuration
# Increase maxconn value
# Adjust timeout parameters
# Enable compression functionality

2. Security Hardening

# Restrict access to the statistics page
# Add ACL rules in haproxy.cfg
acl allowed_ips src 192.168.1.0/24
http-request deny if !allowed_ips

# Enable SSL/TLS
bind *:443 ssl crt /etc/ssl/certs/server.pem
redirect scheme https if !{ ssl_fc }

3. Monitoring and Alerts

# Integrate with a monitoring system
# Configure Prometheus for monitoring
# Set up Grafana dashboards
# Define alert rules

Summary

By combining Keepalived and HAProxy, we successfully built a highly available load balancing cluster. This solution offers the following advantages:

High Availability: Achieved through VRRP protocol for automatic failover.
Load Balancing: Intelligently distributes requests to improve system performance.
Health Checks: Real-time monitoring of service status, automatically removing faulty nodes.
Ease of Maintenance: Simple configuration and convenient management.
Cost Effectiveness: Utilizing open-source software to reduce operational costs.

When deploying in a production environment, it’s also necessary to consider comprehensive aspects such as network security, monitoring and alerts, and backup/restore procedures to ensure stable and reliable operation of the system.