Local Model on Uncle Xiang's Notebook

Writing an AI blog post, in the end, still needs to be turned into engineering (Part 3)

Fri, 03 Apr 2026 21:06:02 +0800

After going through all the configurations in the repository, I am even more certain about one thing: what matters in the end is not how strong any single model is, but rather who should bear the cost at each layer.

The most obvious signal is that the currently active published.runtime.json is still the one generated on April 2, 2026, for minimax-m2, yet the entry from April 3, 2026, at 16:38, labeled 5f17088, has switched the default provider for blog-style-suite to the local gemma-4-26b-a4b in LM Studio. This might look inconsistent, but it actually isn’t; it precisely illustrates that this pipeline has begun to specialize.

With these articles, the first two have laid out the boundaries. The first article discusses why blog-writer emerged, and the second article discusses how blog-style-suite separates style learning from token costs. This final article settles on the most practical question: where should local models, online models, and Minimax ultimately be placed?

Training Style Data, Not Worth Burning Online Models at Every Step

The issue of style data, once you start taking it seriously, quickly becomes a practical problem with tokens. It’s not about whether you want to save costs; if you don’t divide the labor, this whole setup won’t run for long. The most common mistake in the past was letting one online model handle everything.

Scraping historical articles
Performing filtering
Doing categorization
Scoring
Sampling
Enforcing style
Finally writing the draft The biggest problem with doing it this way isn’t that “the model isn’t strong enough,” but rather that every step burns the same level of cost. Looking back now, the truly reasonable approach should be to think in reverse: which steps must be online, which steps should ideally be localized, and which steps shouldn’t even be given to a model at all. As long as this boundary isn’t clear, no matter how powerful the model is, it will just end up helping you repeat a bunch of tasks that could have been pre-processed away.

Local Models are Better Suited for Dirty, Heavy, and Iterative Tasks

I am increasingly inclined to define local models as the “physical layer” for production use. They might not be the strongest, nor perfect every time, but they are particularly suited for tasks such as:

Building through repeated runs/iterations
Multi-round compression experiments on style data
Re-scanning after configuration changes
Low-risk recalculation on existing structures

These types of tasks share a clear commonality. The value isn’t in a single, extremely high-value output, but rather in the ability to run repeatedly, tolerate errors, and ideally avoid paying high costs every single round. Currently, scripts/blog-style-suite/config.json has switched to lm-studio-gemma4, which itself indicates a shift in judgment. It’s not that local gemma is necessarily stronger than online models, but for the production pipeline, we are finally starting to prioritize “runnability, frequency of use, and ability to iterate/modify repeatedly.” This point actually aligns with the logic I wrote previously in Don’t force strong tasks onto weak models. Local models might not be suitable for writing complex, comprehensive articles from scratch, but they are excellent for handling dirty, heavy, and batch processing tasks. Preprocessing style data is inherently more like this category of task.

Online models are better suited for the final polish, not for doing everything from scratch

Just because local models are suitable for the production side doesn’t mean online models have no value. The real value of an online model lies precisely in that final polishing touch. For example:

Supplementing facts based on the latest information
Structuring arguments within a larger context
Handling time-sensitive information that requires internet verification
Transforming already prepared structured style assets into a publishable article These tasks require higher demands on expression quality, factual integration, and contextual understanding, making online models more valuable here. In other words, the powerful model is more like the final few assembly line steps. It’s not that it can’t do more upfront work, but if you make it scan from beginning to end, the entire cost structure will quickly become distorted. This is also why blog-writer is designed to only read from the published location published.runtime.json, rather than having to switch providers or re-scan the suite directory while drafting. The lighter the consumption side, the better it is for a more powerful model to focus on finalizing the article.

The Significance of Minimax: It’s Not Just Another Provider Connection

Many people who see Minimax might first think: “It’s just another model being connected.”

I don’t think so.

The truly valuable aspect of Minimax is that it has successfully paved the way for “multiple provider outputs consumed by a single publishing contract.”

The change on April 2, 2026, at 10:18 (9f15199) modified blog-style-suite to support multi-model configurations, with outputs isolated per provider. Subsequently, the README and runtime structure have consistently emphasized one thing: while the suite can generate many sets of results, only the manually selected published.runtime.json is actually effective.

This boundary is extremely important.

Because once this boundary is clear, the role of Minimax changes from being “something that must be bound within the drafting process” to becoming:

Something that can participate in production-side comparisons.
Something that can be used to generate a runtime version.
Something that can be compared horizontally with local model artifacts.
Finally, something whose publication is decided by human judgment.

This transforms the provider from a “system dependency” into a “replaceable component.”

I believe this is the most interesting significance of Minimax within this engineering setup. It isn’t here to dominate the entire pipeline; it’s here to validate whether this pipeline has successfully cleaned up its interfaces.

True specialization is not based on model strength, but on task type

I now favor a classification method that is quite rudimentary, but very effective.

Rules and Hard Constraints

Leave to local scripts. If it can be solved with deterministic tools like scanner.py, write_post.py, or write_post_series.py, don’t let the model get involved.

Style Data Generation

Prioritize local models or lower-cost providers. Because what is most important here is reproducibility, room for iteration/error, and cacheability, not necessarily the most dazzling single output.

Final Drafting and Fact Consolidation

Hand this off to a model better suited for long-context integration, expression consolidation, and fact-checking/web retrieval. This layer is where spending money on online models is most worthwhile. When broken down like this, many previously confusing issues are actually not that complex. You don’t need to argue every day about “which model is the strongest”; you just need to ask: which layer does this task belong to?

Ultimately, what is most valuable is not the model, but the clear boundaries.

This concludes my third article. As blog-writer and blog-style-suite have evolved, I feel that what is most valuable is not which provider we connected next, or who we replaced, or which one we tested. What is most valuable is that the boundaries are finally becoming clearer.

blog-writer handles the consumption side.
blog-style-suite handles the production side.
published.runtime.json is the publishing point.
Local models are better suited for dirty and heavy lifting that needs to be run repeatedly.
Online models are better suited for the final polish/wrap-up.
Online providers like Minimax feel more like replaceable components rather than the central hub of the system. Once the boundaries are clear, the entire workflow flows smoothly. You won’t expect one model package to conquer everything, nor will you stack every step onto the most expensive layer. In the end, while it looks like selecting a model, what we are actually doing is assigning workstations for different types of tasks. Simply put, having a single strong point is certainly good. But in the long run, clear boundaries are often more important and stronger than any single point solution.

References

Repository Commit: 9f1519967981c5eef7bd1eb407b0406ac542ebd0
Repository Commit: 5f17088391ee858b88fc50df884bc0103ff0b3c1
Repository File: scripts/blog-style-suite/config.json
Effective Runtime: .agents/data/blog-writing/published.runtime.json
Related Old Article: A Period of Heavy AI Programming
Related Old Article: Ultimately Returning to Domestic Models
Related Old Article: Don’t Force Strong Tasks with Weak Models

Writing Notes

Original Prompt

$blog-writer This content is quite extensive, so I've split it into a series of articles: Last year, many drafts were written using large models. Back then, the process was to create an outline or a list of questions myself, and then have the AI generate the draft, copy the content into a local md document, fill in header information, tag information, and publish the article; recently, I used Codex a lot and found that its web search capability is very strong. So, could I write a skill to automate these tasks? This led to the first draft of the skill blog-writer. I also thought about having the AI learn my previous writing style, which caused blog-writer to consume a lot of tokens when running. Subsequently, I optimized blog-writer in several versions, splitting out the data module and the data generation module. The original data generation module was still an independent skill. As I continued writing, I realized that it would be better as a Python project, which led to blog-style-suite. Then, I found that training on style data also consumes a lot of tokens, so I wanted to use a local large model and connected to a local LLM. I then thought about comparing the differences between the local LLM and the online version, so I integrated minimax; the evolution history of blog-style-suite and blog-writer can be analyzed from the git commit history. Additionally, based on the code for local blog-writer and blog-style-suite, I can discuss the design ideas, how token saving was achieved, and how the data structure was designed—the core design concepts. If tokens are abundant, it can consume entire historical articles; preprocessing can save a lot of tokens.

Writing Strategy Summary

The third article will no longer repeat the discussion on architecture, but instead focus solely on the practical issue of “model specialization/division of labor.”
Start directly by stating the current reality—whether to use published.runtime.json from the current repository or if it’s switched locally to gemma4 via minimax-m2 or config.json—to reduce filler content.
The focus should not be on proving which model is stronger, but rather on explaining why different tasks should be assigned to different cost layers.
Placing Minimax in the “replaceable provider” section aims to pull its significance back into the engineering boundary, rather than treating it as just another entry on a model leaderboard.
Conclude by returning to the overarching judgment: “Clear boundaries are more important than single points of strength,” serving as the closing statement for the entire series of articles.

Don't force weak models onto hard tasks.

Thu, 02 Apr 2026 22:05:00 +0800

Recently, I’ve been migrating some edge cases to MiniMax and local models. The more I use them, the more I feel that we shouldn’t always measure things by the standard of “the most powerful model.”

My judgment is straightforward: don’t force weak models into hard tasks. Models like MiniMax are indeed limited in capability, but for complex coding, long-chain reasoning, or ambiguous requirement decomposition, they fall a bit short. However, if you ask it to do data cleaning, document writing, or searching for proposal materials—these kinds of tasks—it can handle them perfectly well. The same logic applies to local models around the 12B size; translation, format rewriting, and batch cleaning are actually where they are best suited.

To put it plainly, it’s not that the models lack value; it’s just that we shouldn’t place them in the wrong roles.

The real problem isn’t how strong the model is, but whether it works correctly.

Many people who talk about large models automatically think of the most difficult tasks.

Writing complex engineering code independently
Deconstructing an entire system in one go
Multi-turn reasoning over long contexts
Planning and executing while searching These are certainly important. But in real-world work, what is actually piled on your desk most often isn’t these kinds of tasks. It’s more like:
Cleaning up a pile of dirty fields
Organizing scattered information into readable documents
Converting long texts into summaries, FAQs, or outlines
Standardizing mixed Chinese and English content formats
Gathering data from multiple web pages and then compiling it into a draft proposal For these types of tasks, what is most needed is not “the model thinking like a genius,” but three things:
Instruction following must be reasonably accurate.
Output structure should be as stable as possible.
The cost must be low enough that you are willing to use it repeatedly. This is why I always feel that weak models are not useless; they just cannot be used in the same kind of battle as flagship models.

MiniMax: What’s Actually Suitable for It

First, let’s talk about MiniMax. The official positioning of MiniMax-M2.5 is actually quite high. In press releases and open platform documentation, they push it towards scenarios like programming, tool calling, search, and office productivity, even emphasizing speed and cost advantages. I don’t completely disbelieve these claims, but I prefer to break them down. For me, what MiniMax is genuinely good at isn’t “the most complex development tasks,” but rather the following:

Data Cleaning

A lot of data cleaning is essentially manual labor involving semi-structured text.

Name unification
Field mapping
Anomaly labeling
Classification tagging
Table field completion What these types of tasks fear most is not the model being “dumb,” but rather inconsistent formatting or divergent outputs. As long as the model can reliably output results in JSON, tables, or fixed templates, it’s actually sufficient. While powerful models certainly can do this, using the most expensive tier of model just to clean fields is often not cost-effective.

Documentation Writing

Writing documentation is annoying, not difficult. When an interface changes, a process changes, or a field is modified, the documentation has to change accordingly. This process doesn’t actually require the model to have strong creativity; rather, it requires it not to over-exert itself and alter clearly defined things into something ambiguous. MiniMax is often more reliable for these kinds of tasks than one might expect. Especially when you have already prepared the context, it acts more like a capable documentation assistant rather than an actual engineer.

Solution Material Search

The official platform is also promoting search and tool calling, so this direction is fine. Many times, what we need is not for the model to “come up with an answer out of thin air,” but rather for it to first find relevant web pages, documents, announcements, or materials, and then organize them neatly. In this scenario, cheaper models like MiniMax are very valuable because searching, summarizing, and integrating are inherently high-frequency, mundane tasks. So my actual view is: MiniMax isn’t incapable; rather, it is better suited for the dirty, tiring, and repetitive tasks within a production pipeline. If you let it act as an assistant or general laborer, it is often competent; but if you ask it to handle the entire engineering process, the probability of disappointment increases.

Local 12B Models, Best Suited for Bringing Back These Tasks

Looking further down, the logic for local deployment is actually the same. When many people talk about local models, they inevitably ask one question: Can it replace the flagship cloud models? I think this question is flawed from the start. For local models around 12B, what has real practical value isn’t “proving that it can handle the most powerful tasks,” but rather bringing back those stable, repetitive, sensitive, low-profit, yet high-frequency tasks.

Translation

This is one of the most natural scenarios for local models. As explicitly mentioned in the official blog of Qwen2.5, it has enhanced capabilities for long-text generation, structured data understanding, and JSON output, and supports over 29 languages. This combination is inherently suitable for tasks like translation, bilingual rewriting, format standardization, and terminology normalization. Technical documentation, field descriptions, product introductions, and API comments—these items often have stable structures and fixed terminology. While local models might not produce the most elegant translations, they are usually sufficient.

Data Cleaning

This is also where local models are particularly realistic. Many spreadsheets, documents, and business materials that you might not want to upload to the cloud. Especially internal data, customer records, meeting minutes, and draft proposals—when privacy and permissions are involved, running it locally provides much more peace of mind. At this point, the significance of a local model around 12B isn’t “how smart it is,” but rather that “it’s on my machine, and it can reliably handle these dirty tasks.”

Fixed Format Rewriting

For example:

Meeting minutes organized into a fixed template
Product titles cleaned into a unified naming convention
Bug descriptions rewritten into ticket format
Mixed Chinese and English text cleaned into single-language versions

These types of tasks share consistent characteristics: clear rules, large batches, high repetition, low value per instance, but significant cumulative effort. This is exactly what local models are best suited for.

Can the 3060 12GB actually run a model around 12B?

I prefer to write about this realistically: “It can run it, but don’t get your hopes up too high.” Google provided a very useful VRAM table in the official documentation for Gemma 3. The Gemma 3 12B roughly requires:

About 20 GB of VRAM to load the full precision version.
About 12.2 GB to load the medium quantization version.
About 8.7 GB to load a lower VRAM consumption version. The official documentation also specifically reminds that this is only for model loading, and does not include prompt or runtime overhead. This sentence is very key. What does it mean? It means that running a model around 12B on a card like the 3060 12GB is not impossible, but the prerequisites are usually:
You are running a quantized version.
The context length should not be too long.
The task shouldn’t be too complex.
You accept average, or even slow, speed. If you are willing to accept these premises, then running a local 12B model is indeed feasible. Tasks like translation, summarization, table cleaning, and fixed format conversion are not exaggerated in this regard. Furthermore, the official repository for Qwen2.5-14B-Instruct-GGUF itself provides multiple quantization formats, which actually makes the intention very clear: models in this category are inherently adapted for the local inference ecosystem. So my conclusion has never been that “the 3060 12GB can easily handle a 12B model,” but rather: It can run these types of models, but it is better suited for work with low expectations, high repetition, and high privacy requirements.

Cheap Models and Local Models: It’s Not Just About Saving API Costs

When people talk about this, the first reaction is always saving money. Of course, saving money is important. But I think the greater value is that you start daring to outsource all those little tasks you used to avoid doing. Before, you might not have written a dedicated script just to clean up a few hundred data points. You also wouldn’t manually adjust dozens of pages of mixed Chinese and English documents to achieve uniform formatting. And you certainly wouldn’t read through every single webpage to gather materials for an ad-hoc proposal. Things are different now. As long as the cost is low enough and the barrier is low enough, these tasks that were previously considered “not worth the effort” suddenly become worthwhile. You no longer hesitate over whether or not to do it; instead, you just throw it to a cheap model or a local model to run through first. This is what I see as the most realistic change. Powerful models are responsible for tackling core problems, weaker models handle miscellaneous tasks, and local models provide fallback and batch processing. With this division of labor, the entire workflow becomes smooth.

Conclusion

So, the final word remains: don’t always try to make one model conquer everything. Models like MiniMax are weak in capability, but they aren’t useless. If you use them to tackle complex engineering tasks, vague requirements, or multi-turn reasoning, you will naturally be disappointed; however, if you use them for data cleaning, document drafting, or searching for proposal materials, they often work quite smoothly. The same applies to local models around 12B. Their purpose isn’t to prove that “I no longer need cloud flagships,” but rather to reliably move stable, repetitive, sensitive, and high-volume tasks back onto their own machines. Simply put: don’t let a weak model do what it is not good at. Place them in the right role, and they will have real value.

References

Writing Notes

Original Prompt

Minimax’s large model is weak in capability, but it’s fine for tasks like data cleaning, document writing, and searching for proposal materials; with the same logic, deploying a large model locally for translation or data cleaning work is also good. The model parameter count is around 12b, and even a local GPU like the RTX 3060 with 12GB can handle it.

Writing Outline Summary

Retained the core judgment of “don’t force weak models onto hard tasks,” and did not write it as a model leaderboard comparison.
The MiniMax section is mainly based on the official positioning for programming, searching, and office work, then applies this judgment back to real-world tasks like data cleaning, document handling, and information retrieval.
For local models, I selected two officially sourced options: Qwen2.5 and Gemma 3, one supporting multilingual and structured output, and the other supporting 12B size and VRAM usage.
The description for the 3060 12GB was intentionally phrased as “capable, but don’t get too carried away,” to avoid presenting quantized inference as an absolute conclusion.
In the conclusion, I re-categorized strong models, weak models, and local models based on their respective roles, making the main thread more focused.