Add sync-models script, server.py reference proxy, and doc updates
This commit is contained in:
Nguyen Hong Nhat
parent
e974a37475
commit
9bcb4b43e1
@ -26,9 +26,11 @@ npm install # dev deps: typescript, @types/node, @ai-sdk/provider
|
||||
npm run typecheck # tsc --noEmit
|
||||
npm run build # tsc -> dist/
|
||||
npm run smoke # live request against CommandCode (needs a key)
|
||||
npm run sync-models # regenerate provider.commandcode.models from the live catalog
|
||||
```
|
||||
|
||||
There is no unit-test suite. `scripts/smoke.mjs` is the end-to-end check.
|
||||
There is no unit-test suite. `scripts/smoke.mjs` is the end-to-end check. `scripts/sync-models.mjs`
|
||||
is the model-catalog generator (needs `dist/` first; writes the opencode config unless `--out`/`--dry-run`).
|
||||
|
||||
After any change under `src/`, run `npm run typecheck`, then `npm run build`, then `npm run smoke`.
|
||||
A change is not done until it typechecks and the smoke test passes.
|
||||
@ -44,6 +46,7 @@ src/usage.ts finish event -> LanguageModelV3Usage; finish-reason unificatio
|
||||
src/redact.ts Credential scrubbing for error surfaces.
|
||||
src/constants.ts Defaults, paths, headers, passthrough params, static config block.
|
||||
scripts/smoke.mjs Live end-to-end check.
|
||||
scripts/sync-models.mjs Catalog -> provider.commandcode.models generator (with vision probing).
|
||||
server.py Reference Python proxy (do not modify unless explicitly asked).
|
||||
```
|
||||
|
||||
@ -101,6 +104,10 @@ These are load-bearing. Breaking one causes silent failures in opencode.
|
||||
`redact()`.
|
||||
9. **`file://` npm specs bypass install.** opencode imports `dist/index.js` directly, so the repo
|
||||
must be rebuilt for opencode to see source changes.
|
||||
10. **`/models` is config-driven, not provider-driven.** opencode builds the model list from
|
||||
`provider.commandcode.models` in `opencode.json` and never asks a custom `file://` provider to
|
||||
discover models (only internal providers can register `discoverModels`). Refresh the map with
|
||||
`npm run sync-models`; do not expect a discovery hook in `src/` to populate it.
|
||||
|
||||
## Change workflow
|
||||
|
||||
|
||||
52
README.md
52
README.md
@ -76,8 +76,54 @@ opencode models commandcode
|
||||
> a rebuilt provider, quit and relaunch opencode for the change to take effect.
|
||||
|
||||
The `models` map is required — opencode silently drops a custom provider whose model list is
|
||||
empty. List every model you want selectable; the provider itself only needs `languageModel(id)`,
|
||||
which it implements for any id passed to it.
|
||||
empty. The provider itself only needs `languageModel(id)`, which it implements for any id passed
|
||||
to it, but opencode builds `/models` and the TUI picker from this map and never asks a custom
|
||||
`file://` provider to discover models. Use the sync script below to fill it in.
|
||||
|
||||
## Listing all CommandCode models
|
||||
|
||||
opencode reads the model list from `provider.commandcode.models`; a custom provider cannot
|
||||
register dynamic discovery. `scripts/sync-models.mjs` fetches the live catalog from
|
||||
`GET /provider/v1/models` and writes the full map into your opencode config.
|
||||
|
||||
```powershell
|
||||
npm run build # the script imports the compiled provider
|
||||
npm run sync-models # fetch catalog, probe vision, merge into ~/.config/opencode/opencode.json
|
||||
```
|
||||
|
||||
Then restart opencode and confirm:
|
||||
|
||||
```powershell
|
||||
opencode models commandcode
|
||||
```
|
||||
|
||||
Mapping and behaviour:
|
||||
|
||||
- Every catalog entry gets `limit.context` from its `context_length`, `limit.output` from
|
||||
`DEFAULT_MAX_TOKENS`, and **`reasoning: true`**.
|
||||
- Vision (`attachment` + `modalities.input` with `image`) is **probed** per model: a 64×64 image
|
||||
is sent and the reply is inspected. Models that answer a color are marked vision; models that
|
||||
reply `NO_IMAGE` (retried once) or reject the image are not. Probes that cannot run — plan-gated,
|
||||
temporarily unavailable — fall back to a family heuristic (`claude`, `gpt-5`, `gemini`, `grok`,
|
||||
`qwen…vl`, …). Probing is best-effort: upstream is nondeterministic and a re-run may flip a
|
||||
borderline model.
|
||||
- Generated fields (`name`, `limit`, `reasoning`, `attachment`, `modalities`) overwrite existing
|
||||
values. Other fields (`variants`, `options`, `cost`, …) and ids not in the catalog are preserved.
|
||||
|
||||
Options:
|
||||
|
||||
```powershell
|
||||
node scripts/sync-models.mjs --dry-run # print the merged map, write nothing
|
||||
node scripts/sync-models.mjs --out models.json # write only the models fragment
|
||||
node scripts/sync-models.mjs --no-probe # skip probing, use the heuristic (fast/offline)
|
||||
node scripts/sync-models.mjs --concurrency 8 # probe parallelism (default 6)
|
||||
node scripts/sync-models.mjs --no-preserve # snapshot only: drop user ids/extra fields
|
||||
node scripts/sync-models.mjs --config <path> # target a different config
|
||||
```
|
||||
|
||||
The API key is read from `COMMANDCODE_API_KEY`, then `provider.commandcode.options.apiKey` in the
|
||||
target config.
|
||||
|
||||
|
||||
## Configuration options
|
||||
|
||||
@ -148,6 +194,7 @@ src/usage.ts finish event -> V3 usage; finish-reason unification.
|
||||
src/redact.ts Credential scrubbing for error surfaces.
|
||||
src/constants.ts Defaults, header names, passthrough params, static config block.
|
||||
scripts/smoke.mjs Live end-to-end check against api.commandcode.ai.
|
||||
scripts/sync-models.mjs Generate provider.commandcode.models from the live catalog.
|
||||
```
|
||||
|
||||
### Request lifecycle
|
||||
@ -175,6 +222,7 @@ npm install # installs typescript, @types/node, @ai-sdk/provider
|
||||
npm run typecheck # tsc --noEmit
|
||||
npm run build # tsc -> dist/
|
||||
npm run smoke # live request against CommandCode
|
||||
npm run sync-models # regenerate provider.commandcode.models from the catalog
|
||||
```
|
||||
|
||||
`scripts/smoke.mjs` reads the key from `COMMANDCODE_API_KEY`, falling back to
|
||||
|
||||
@ -17,7 +17,8 @@
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"smoke": "node scripts/smoke.mjs"
|
||||
"smoke": "node scripts/smoke.mjs",
|
||||
"sync-models": "node scripts/sync-models.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@ai-sdk/provider": "3.0.8",
|
||||
|
||||
312
scripts/sync-models.mjs
Normal file
312
scripts/sync-models.mjs
Normal file
@ -0,0 +1,312 @@
|
||||
// Generate the opencode `provider.commandcode.models` map from CommandCode's live catalog.
|
||||
//
|
||||
// opencode builds `/models` from the configured model map; a custom `file://` provider is only
|
||||
// ever asked for `languageModel(id)`, never queried for discovery. So the catalog has to be baked
|
||||
// into opencode.json. This script fetches it, probes each model for vision support, and merges the
|
||||
// result into the target config.
|
||||
//
|
||||
// Run: node scripts/sync-models.mjs [options] (build first: npm run build)
|
||||
import { readFileSync, writeFileSync, renameSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createCommandCode } from "../dist/index.js";
|
||||
import { redact } from "../dist/redact.js";
|
||||
import { DEFAULT_BASE_URL, DEFAULT_CC_VERSION, DEFAULT_MAX_TOKENS, MODELS_PATH } from "../dist/constants.js";
|
||||
|
||||
// 64x64 solid red PNG, used only to provoke a vision-accepting vs vision-rejecting response.
|
||||
const PROBE_IMAGE = {
|
||||
type: "file",
|
||||
mediaType: "image/png",
|
||||
data: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAIAAAAlC+aJAAAAeUlEQVR4nO3PQQkAMAzAwCqpf1ETMxF7HINABFzm7H7dcEEDWtCAFjSgBQ1oQQNa0IAWNKAFDWhBA1rQgBY0oAUNaEEDWtCAFjSgBQ1oQQNa0IAWNKAFDWhBA1rQgBY0oAUNaEEDWtCAFjSgBQ1oQQNa0IAWNKAFj13PLIEAOXyUUwAAAABJRU5ErkJggg==",
|
||||
};
|
||||
|
||||
const VISION_ERROR = /image|vision|multimodal|modality|unsupported|not support|invalid.*content|does not support/i;
|
||||
// Text-only models answer the image prompt by admitting they cannot see an image. Plan-gated or
|
||||
// unavailable models error before that, so only a clean reply is treated as a vision signal.
|
||||
const NO_IMAGE =
|
||||
/(?:don'?t|do not|can'?t|cannot|unable to)\s+(?:see|view|detect|find|access|receive|open|process)|no\s+(?:image|picture|attachment|photo)|not\s+(?:see|receive|detect|attach)|image\s+(?:was\s+)?not\s+(?:attach|provid|receiv|includ)/i;
|
||||
|
||||
// Fallback when a probe is inconclusive or disabled. Ground truth is unavailable from the catalog,
|
||||
// so this is only a best guess for ids that look like known vision families.
|
||||
const VISION_HEURISTIC = /claude|gpt-5|gemini|grok|qwen.*vl|vision|(?:^|[^a-z])vl(?:[^a-z]|$)|omni|multimodal/i;
|
||||
|
||||
function parseArgs(argv) {
|
||||
const opts = {
|
||||
config: join(homedir(), ".config", "opencode", "opencode.json"),
|
||||
baseURL: undefined,
|
||||
out: undefined,
|
||||
dryRun: false,
|
||||
probe: true,
|
||||
concurrency: 6,
|
||||
preserve: true,
|
||||
};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const arg = argv[i];
|
||||
const value = () => {
|
||||
const next = argv[++i];
|
||||
if (next === undefined) throw new Error(`Missing value for ${arg}`);
|
||||
return next;
|
||||
};
|
||||
switch (arg) {
|
||||
case "--config": opts.config = value(); break;
|
||||
case "--base-url": opts.baseURL = value(); break;
|
||||
case "--out": opts.out = value(); break;
|
||||
case "--concurrency": opts.concurrency = Math.max(1, Number(value()) || 1); break;
|
||||
case "--dry-run": opts.dryRun = true; break;
|
||||
case "--no-probe": opts.probe = false; break;
|
||||
case "--no-preserve": opts.preserve = false; break;
|
||||
case "-h":
|
||||
case "--help":
|
||||
console.log(
|
||||
[
|
||||
"Generate provider.commandcode.models from the CommandCode catalog.",
|
||||
"",
|
||||
"Usage: node scripts/sync-models.mjs [options]",
|
||||
" --config <path> opencode config to update (default ~/.config/opencode/opencode.json)",
|
||||
" --base-url <url> upstream origin (default from config or api.commandcode.ai)",
|
||||
" --out <file> write only the models fragment; do not touch the config",
|
||||
" --dry-run print the merged map; write nothing",
|
||||
" --no-probe skip vision probing; use the family heuristic",
|
||||
" --concurrency <n> vision probe parallelism (default 6)",
|
||||
" --no-preserve snapshot only; drop user-added ids and extra fields",
|
||||
].join("\n"),
|
||||
);
|
||||
process.exit(0);
|
||||
break;
|
||||
default:
|
||||
throw new Error(`Unknown argument: ${arg}`);
|
||||
}
|
||||
}
|
||||
return opts;
|
||||
}
|
||||
|
||||
function readConfig(path) {
|
||||
const text = readFileSync(path, "utf8");
|
||||
return { text, data: JSON.parse(text) };
|
||||
}
|
||||
|
||||
function detectIndent(text) {
|
||||
const match = /\n([ \t]+)\S/.exec(text);
|
||||
if (!match) return 2;
|
||||
const indent = match[1];
|
||||
if (indent.startsWith("\t")) return "\t";
|
||||
return indent.length;
|
||||
}
|
||||
|
||||
function resolveKey(config, env) {
|
||||
const fromEnv = env.COMMANDCODE_API_KEY;
|
||||
if (fromEnv) return fromEnv.trim();
|
||||
const options = config?.provider?.commandcode?.options ?? {};
|
||||
const fromConfig = options.apiKey;
|
||||
if (typeof fromConfig === "string" && fromConfig.trim()) return fromConfig.trim();
|
||||
const auth = options.headers?.Authorization ?? options.headers?.authorization;
|
||||
if (typeof auth === "string" && auth.trim()) return auth.replace(/^Bearer\s+/i, "").trim();
|
||||
return undefined;
|
||||
}
|
||||
|
||||
async function fetchCatalog(baseURL, apiKey, ccVersion) {
|
||||
const response = await fetch(`${baseURL}${MODELS_PATH}`, {
|
||||
headers: {
|
||||
Authorization: /^Bearer\s/i.test(apiKey) ? apiKey : `Bearer ${apiKey}`,
|
||||
"x-command-code-version": ccVersion,
|
||||
},
|
||||
});
|
||||
const body = await response.text();
|
||||
if (!response.ok) {
|
||||
throw new Error(`catalog ${response.status}: ${redact(body.slice(0, 500)) || response.statusText}`);
|
||||
}
|
||||
const parsed = JSON.parse(body);
|
||||
const data = Array.isArray(parsed?.data) ? parsed.data : [];
|
||||
return data.filter((m) => m && typeof m.id === "string" && m.id.length > 0);
|
||||
}
|
||||
|
||||
async function probeVision(provider, id) {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(new Error("probe timeout")), 30000);
|
||||
try {
|
||||
const result = await provider.languageModel(id).doGenerate({
|
||||
prompt: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
"Look at the attached image and reply with exactly one word: its dominant color. " +
|
||||
"If no image is attached, or you cannot see images, reply with exactly: NO_IMAGE",
|
||||
},
|
||||
PROBE_IMAGE,
|
||||
],
|
||||
},
|
||||
],
|
||||
maxOutputTokens: 256,
|
||||
temperature: 0,
|
||||
abortSignal: controller.signal,
|
||||
});
|
||||
const text = (result.content ?? [])
|
||||
.map((part) => (part.type === "text" ? part.text : ""))
|
||||
.join("")
|
||||
.trim();
|
||||
if (!text) return { vision: undefined, reason: `empty (${result.finishReason?.unified ?? "?"})` };
|
||||
if (/NO[_\s-]?IMAGE/i.test(text) || NO_IMAGE.test(text)) return { vision: false, reason: "reported no image" };
|
||||
return { vision: true, reason: `answered: ${text.slice(0, 24)}` };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
if (VISION_ERROR.test(message)) return { vision: false, reason: "rejected image" };
|
||||
return { vision: undefined, reason: redact(message).slice(0, 120) || "error" };
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
// Some models intermittently deny seeing an image on the first try. Retry an explicit "no image"
|
||||
// once so a flaky denial does not mislabel a vision model; inconclusive results skip the retry.
|
||||
async function probeVisionConfirmed(provider, id) {
|
||||
let outcome = await probeVision(provider, id);
|
||||
if (outcome.vision === false) outcome = await probeVision(provider, id);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
async function mapPool(items, concurrency, worker) {
|
||||
const results = new Array(items.length);
|
||||
let cursor = 0;
|
||||
const runners = Array.from({ length: Math.min(concurrency, items.length) }, async () => {
|
||||
while (true) {
|
||||
const index = cursor++;
|
||||
if (index >= items.length) return;
|
||||
results[index] = await worker(items[index], index);
|
||||
}
|
||||
});
|
||||
await Promise.all(runners);
|
||||
return results;
|
||||
}
|
||||
|
||||
function modelEntry(catalogModel, vision) {
|
||||
const id = catalogModel.id;
|
||||
const context = Number(catalogModel.context_length) || 0;
|
||||
const entry = {
|
||||
name: typeof catalogModel.name === "string" && catalogModel.name ? catalogModel.name : id,
|
||||
limit: { context, output: DEFAULT_MAX_TOKENS },
|
||||
reasoning: true,
|
||||
attachment: vision,
|
||||
modalities: {
|
||||
input: vision ? ["text", "image"] : ["text"],
|
||||
output: ["text"],
|
||||
},
|
||||
};
|
||||
return entry;
|
||||
}
|
||||
|
||||
function mergeModels(existing, generated, preserve) {
|
||||
const out = {};
|
||||
if (preserve) {
|
||||
for (const [id, model] of Object.entries(existing ?? {})) out[id] = model;
|
||||
}
|
||||
for (const [id, entry] of Object.entries(generated)) {
|
||||
out[id] = preserve ? { ...(existing?.[id] ?? {}), ...entry } : entry;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function writeAtomic(path, contents) {
|
||||
const tmp = `${path}.${process.pid}.tmp`;
|
||||
writeFileSync(tmp, contents);
|
||||
renameSync(tmp, path);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs(process.argv.slice(2));
|
||||
const { text, data: config } = readConfig(opts.config);
|
||||
|
||||
const key = resolveKey(config, process.env);
|
||||
if (!key) {
|
||||
console.error("No API key: set COMMANDCODE_API_KEY or provider.commandcode.options.apiKey");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const baseURL = (opts.baseURL ?? config?.provider?.commandcode?.options?.baseURL ?? DEFAULT_BASE_URL).replace(/\/+$/, "");
|
||||
const ccVersion = config?.provider?.commandcode?.options?.ccVersion ?? DEFAULT_CC_VERSION;
|
||||
|
||||
console.log(`Fetching catalog: ${baseURL}${MODELS_PATH}`);
|
||||
let catalog;
|
||||
try {
|
||||
catalog = await fetchCatalog(baseURL, key, ccVersion);
|
||||
} catch (error) {
|
||||
console.error(`Failed to fetch catalog: ${error instanceof Error ? error.message : String(error)}`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(`Catalog: ${catalog.length} models`);
|
||||
|
||||
const provider = createCommandCode({ name: "commandcode", apiKey: key, baseURL, ccVersion, maxRetries: 1 });
|
||||
|
||||
let probeResults = [];
|
||||
if (opts.probe) {
|
||||
console.log(`Probing vision (${catalog.length} models, concurrency ${opts.concurrency})...`);
|
||||
probeResults = await mapPool(catalog, opts.concurrency, async (model, index) => {
|
||||
const outcome = await probeVisionConfirmed(provider, model.id);
|
||||
const tag = outcome.vision === true ? "vision" : outcome.vision === false ? "no-vision" : "unknown";
|
||||
console.log(` [${index + 1}/${catalog.length}] ${model.id}: ${tag} (${outcome.reason})`);
|
||||
return outcome;
|
||||
});
|
||||
}
|
||||
|
||||
const generated = {};
|
||||
const stats = { probed: 0, vision: 0, noVision: 0, unknown: 0, heuristic: 0 };
|
||||
catalog.forEach((model, index) => {
|
||||
const outcome = probeResults[index];
|
||||
let vision;
|
||||
if (outcome && outcome.vision !== undefined) {
|
||||
vision = outcome.vision;
|
||||
stats.probed++;
|
||||
if (vision) stats.vision++;
|
||||
else stats.noVision++;
|
||||
} else {
|
||||
vision = VISION_HEURISTIC.test(`${model.id} ${model.name ?? ""}`);
|
||||
stats.unknown++;
|
||||
if (vision) stats.heuristic++;
|
||||
}
|
||||
generated[model.id] = modelEntry(model, vision);
|
||||
});
|
||||
|
||||
const existing = config?.provider?.commandcode?.models ?? {};
|
||||
const merged = mergeModels(existing, generated, opts.preserve);
|
||||
const added = Object.keys(generated).filter((id) => !(id in existing)).length;
|
||||
const preservedIds = Object.keys(existing).filter((id) => !(id in generated)).length;
|
||||
|
||||
if (opts.out) {
|
||||
writeAtomic(opts.out, `${JSON.stringify(generated, null, 2)}\n`);
|
||||
console.log(`Wrote ${Object.keys(generated).length} models to ${opts.out}`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (opts.dryRun) {
|
||||
console.log(JSON.stringify(merged, null, 2));
|
||||
console.log(
|
||||
`\n[dry-run] generated=${Object.keys(generated).length} added=${added} ` +
|
||||
`preserved-ids=${preservedIds} probed=${stats.probed} vision=${stats.vision} ` +
|
||||
`no-vision=${stats.noVision} unknown=${stats.unknown} heuristic-vision=${stats.heuristic}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
config.provider ??= {};
|
||||
config.provider.commandcode ??= {};
|
||||
config.provider.commandcode.models = merged;
|
||||
|
||||
const indent = detectIndent(text);
|
||||
const trailing = text.endsWith("\n") ? "\n" : "";
|
||||
writeAtomic(opts.config, `${JSON.stringify(config, null, indent)}${trailing}`);
|
||||
|
||||
console.log(
|
||||
`Wrote ${Object.keys(merged).length} models to ${opts.config}\n` +
|
||||
` generated=${Object.keys(generated).length} added=${added} preserved-ids=${preservedIds}\n` +
|
||||
` probed=${stats.probed} vision=${stats.vision} no-vision=${stats.noVision} ` +
|
||||
`unknown=${stats.unknown} heuristic-vision=${stats.heuristic}\n` +
|
||||
`Restart opencode, then run: opencode models commandcode`,
|
||||
);
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error(error instanceof Error ? error.stack : String(error));
|
||||
process.exit(1);
|
||||
});
|
||||
Loading…
Reference in New Issue
Block a user