Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/architecture/generative-ui-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

新回复可以保存为 `openbitfun://current-session/artifacts/visualizations/v2/<slug>.html`。FlowChat 按原始内容引用中的 URI 找到同一轮、同一子 Agent 的成功 Write,使用调用中保存的字节创建隔离的 iframe。同轮 Edit 在回复发出前生效。执行端在成功的 Write 和 Edit 结果中记录会话 URI;通过该会话本地存储目录的绝对路径修改图时,也会带上同一 URI。新的 Edit 还保存该次修改的完整源码。执行端完成对话轮次时,将最后一次通过检查的 Write 或 Edit 中的原始源码和 UTF-8 字节数保存在该轮记录中。旧版写入记录沿用完成时的文件快照。通过脚本生成的图也能发布,前提是该轮使用了可视化技能,且被引用的文件在该轮内生成或修改。FlowChat 收到已完成的回复和重开历史消息时都使用这份源码。读取旧回复时,执行端从已记录的 Write 或完整 Edit 开始,按顺序精确回放截至该条回复的修改。后续对话可以只修改已有图,新的回复使用当轮 Edit 返回的完整源码,上一条回复保持修改前的样子。生成成功后,图直接出现在回复里。若模型在最终文本里漏写内容引用,FlowChat 会在已完成的该轮回复中补上有成功记录的图,已引用的图只显示一次。原版图表的 `mountChart`、四个专用组件、地图、插图、变体轮播和设计控件都能使用同一入口。参考图仅用于测试。模型按用户的题目自行编排内容。 FlowChat 将当前聊天区域的颜色和字体通过展示通道传给已打开的图。切换主题或皮肤时,图内使用原版 CSS 变量的标记和控件会随之更新,交互状态留在原来的文档中。

常规构建随源码提供 OpenBitFun 的技能入口、隔离渲染外壳、Apache ECharts 和离线地图。Vite 打包地图投影 worker 及其数据,Rust 嵌入技能。构建入口会通过 SHA-256 检查随仓库分发的依赖和地理数据,开发者也可以运行 `pnpm run visualize:verify` 单独检查。Write 和 Edit 在桌面后台实际渲染并返回本次产物的引用;无图形宿主时检查延至阅读端。Codex Visualize 1.0.103 的本地兼容套件是可选项。已有原包的开发者可以运行 `pnpm run visualize:import-reference` 导入扩展指南及兼容资源。
常规构建随源码提供 OpenBitFun 的技能入口、隔离渲染外壳、Apache ECharts 和离线地图。Vite 打包地图投影 worker 及其数据,Rust 嵌入技能。构建入口会通过 SHA-256 检查随仓库分发的依赖和地理数据,开发者也可以运行 `pnpm run visualize:verify` 单独检查。Write 和 Edit 在桌面后台实际渲染并返回本次产物的引用;无图形宿主时检查延至阅读端,使用产生图表的那条用户请求。若街道图只保存了声明,读者可以选择联网加载底图;成功后,同一修订可在本机离线回放。Codex Visualize 1.0.109 的本地兼容套件是可选项。原包标注为专有许可,Git 仓库记录文件校验值。Rust 的 debug 和 release 构建均排除本机原包和扩展指南。只有桌面开发服务器在本地确认套件可用时才向模型提供扩展指南。按时间规划出行时可使用日历;询问位置、地图或空间关系时可使用地理底图。已有原包的开发者可以运行 `pnpm run visualize:import-reference` 导入扩展指南及兼容资源。

普通城市标注图可以在技能交付的 HTML 中声明地点及可选范围。FlowChat 使用随产品分发的 Natural Earth 底图与地名,将声明投影成 SVG,检查城市标注和底图并保存该次检查的 SVG。断网重开历史图时,地理图层已经包含在回复中。皮肤控制画布和字体,水域、陆地、标记各自使用独立的语义颜色。

Expand Down Expand Up @@ -32,7 +32,7 @@ FlowChat 按会话、轮次和子 Agent 身份查找该段文字之前最后一

## 组件验收

对照 Codex visualize 1.0.103 的 charts 1.6.5 与 widgets 1.0.59 文档,OpenBitFun 使用独立的声明式输入和本地渲染器。可信代码随产品提供。macOS WKWebView 也验证了图表筛选、参数滑块、请求自身耗时、非连续 CPU 调用与刷新恢复。以下用例通过实际组件数据验算以及 Chromium 中的 FlowChat 回复回放检查。可运行的验收位于 [图表断言](../../src/web-ui/src/tools/visualize/components.test.ts) 和 [浏览器用例](../../scripts/test-visualize-browser.mjs)。
当前兼容测试覆盖 Visualize 1.0.109 的 charts 1.8.1、历史图使用的 charts 1.6.5,以及 widgets 1.0.59。OpenBitFun 使用独立的声明式输入和本地渲染器。可信代码随产品提供。macOS WKWebView 也验证了图表筛选、参数滑块、请求自身耗时、非连续 CPU 调用与刷新恢复。以下用例通过实际组件数据验算以及 Chromium 中的 FlowChat 回复回放检查。可运行的验收位于 [图表断言](../../src/web-ui/src/tools/visualize/components.test.ts) 和 [浏览器用例](../../scripts/test-visualize-browser.mjs)。

| 组件 | 核对内容 |
| --- | --- |
Expand Down
6 changes: 6 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions scripts/dev.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

const fs = require('fs');
const net = require('net');
const { visualizeDevGuide } = require('./visualize-dev-kit.cjs');
const { execSync, spawn } = require('child_process');
const path = require('path');
const { pathToFileURL } = require('url');
Expand Down Expand Up @@ -817,6 +818,7 @@ async function main() {
// Point Remote Connect at the live workspace dist so a newly generated
// QR code never uploads a stale mobile-web bundle.
OPENBITFUN_MOBILE_WEB_DIR: path.join(ROOT_DIR, 'src/mobile-web/dist'),
OPENBITFUN_VISUALIZE_DEV_GUIDE: visualizeDevGuide(ROOT_DIR),
};
try {
const args = ['dev', '--config', tauriConfig, '--config', JSON.stringify({ build: { devUrl: `http://localhost:${DEV_SERVER_PORT}` } })];
Expand All @@ -828,10 +830,7 @@ async function main() {
await runWindowsCommandArgs(tauriBin, args, desktopDir, tauriDevEnv);
} else {
const tauriBin = path.join(ROOT_DIR, 'node_modules', '.bin', 'tauri');
await spawnCommand(tauriBin, args, desktopDir, {
CARGO_PROFILE_DEV_CODEGEN_UNITS: tauriDevEnv.CARGO_PROFILE_DEV_CODEGEN_UNITS,
OPENBITFUN_MOBILE_WEB_DIR: tauriDevEnv.OPENBITFUN_MOBILE_WEB_DIR,
});
await spawnCommand(tauriBin, args, desktopDir, tauriDevEnv);
}
} finally {
// Option B: prune only when the desktop:dev session ends, not on each rebuild.
Expand Down
63 changes: 49 additions & 14 deletions scripts/import-visualize.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { cp, lstat, mkdir, mkdtemp, readFile, readdir, rename, rm, writeFile } from 'node:fs/promises';
import { cp, mkdir, mkdtemp, readFile, readdir, rename, rm, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
Expand All @@ -9,6 +9,7 @@ const root = fileURLToPath(new URL('../', import.meta.url));
const manifest = JSON.parse(await readFile(new URL('./visualize-upstream.json', import.meta.url), 'utf8'));
const owner = path.join(root, 'src/crates/assembly/core/builtin_skills/visualize');
const target = path.join(owner, 'upstream');
const cache = path.join(root, '.visualize-runtime');
const args = process.argv.slice(2);
const check = args.includes('--check');
const paths = args.filter((arg) => arg !== '--check');
Expand Down Expand Up @@ -43,28 +44,62 @@ async function verify(directory) {
assert.deepEqual(found.sort(), Object.keys(manifest.files).sort(), 'Missing upstream files');
}

if (!check && paths.length) await verify(source);
try {
await lstat(target);
} catch (e) {
if (check || e.code !== 'ENOENT') throw e;
if (!paths.length) await verify(source);
// Stage alongside the containing skill, outside the embedded skill tree.
const stage = await mkdtemp(path.join(root, '.visualize-import-'));
const installed = await readFile(path.join(target, '.codex-plugin/plugin.json'), 'utf8')
.then(JSON.parse)
.catch(error => {
if (error.code !== 'ENOENT') throw error;
});
if (!check && (paths.length || installed?.version !== manifest.version)) await verify(source);
if (!check && installed?.version !== manifest.version) {
// Keep the previous local install outside the embedded skill tree during upgrades.
await mkdir(cache, { recursive: true });
const stage = await mkdtemp(path.join(cache, 'import-'));
const backup = path.join(stage, 'previous');
let preserveBackup = false;
try {
await cp(source, path.join(stage, 'package'), { recursive: true, preserveTimestamps: true });
await verify(path.join(stage, 'package'));
await rename(path.join(stage, 'package'), target);
if (installed) await rename(target, backup);
try {
await rename(path.join(stage, 'package'), target);
} catch (error) {
if (installed) {
preserveBackup = true;
await rename(backup, target);
preserveBackup = false;
}
throw error;
}
} finally {
await rm(stage, { recursive: true, force: true });
if (!preserveBackup) await rm(stage, { recursive: true, force: true });
}
}
await verify(target);

// Before this importer cleaned its staging directory, a successful upgrade
// left exactly one "previous" install. Leave interrupted swaps and unrelated
// cache entries alone. Only prune after verifying the current install.
if (!check) {
const stages = await readdir(cache, { withFileTypes: true }).catch(error => {
if (error.code === 'ENOENT') return [];
throw error;
});
for (const stage of stages.filter(entry => entry.isDirectory() && /^import-[A-Za-z0-9]{6}$/.test(entry.name))) {
const dir = path.join(cache, stage.name);
const entries = await readdir(dir, { withFileTypes: true }).catch(() => []);
if (entries.length !== 1 || entries[0].name !== 'previous' || !entries[0].isDirectory()) continue;
const plugin = await readFile(path.join(dir, 'previous/.codex-plugin/plugin.json'), 'utf8').then(JSON.parse).catch(() => null);
if (plugin?.name === 'visualize' && typeof plugin.version === 'string' && plugin.version !== manifest.version)
await rm(dir, { recursive: true, force: true });
}
}

// Keep the packaged guides and validation workflow intact. Only the resource
// location and the final reference differ between Codex and FlowChat.
const guidePrefix = 'skills/visualize/';
const uri = 'openbitfun://current-session/artifacts/visualizations/v2/<title>.html';
const streetMaps = 'For a geographic map write to map-<slug>.html. For a map of sightseeing, walking or transit stops within one city, use detail:"street", city, optional ISO country and the ordered public stop names in places. Submit the names directly. The host resolves stops, fetches the visible tiles, and saves the verified map for offline replay. Use a short local query and a matching osmTag (railway:station or highway:pedestrian for those stop types), or a reported osmId to identify the intended place. Ask about unresolved ambiguity. Sourced WGS84 coordinates are also accepted. Do not invent roads, shorelines, route geometry or scales. For a missing-map follow-up write the atlas declaration again in the current turn using the earlier stops. Repair it with the verified candidate coordinates in the current receipt. If name lookup fails for a public stop, get sourced WGS84 coordinates with WebSearch or WebFetch and resubmit the same map with a source credit. If coordinates or tiles remain unavailable, report the current error in text. An itinerary request may still show its schedule as an <ol data-visual-itinerary>. For explicitly offline or private locations use sourced local geometry. For figures with bound data supply stop coordinates or local geometry.';

function flowChatEntry(upstream, resourceDirectory) {
const adaptedMap = resourceDirectory.startsWith('..') ? './maps.md' : './authoring/maps.md';
return upstream
Expand All @@ -78,9 +113,9 @@ function flowChatEntry(upstream, resourceDirectory) {
.replace(/### File[\s\S]*?(?=### Fragment)/,
'### Publish in FlowChat\n\n' +
'- An explicitly requested webpage (网页), website, or project page is a normal project file. Deliver a working independently openable page and link its entry file.\n' +
'- For a conversational figure, write the literal fragment directly with Write. The first line is `+++ ' + uri + '` (replace <title> with an ASCII lowercase slug). The next line starts the markup. This is your only HTML destination. Use Edit on that URI for corrections.\n' +
'- For an inline HTML figure, write the literal fragment directly with Write. The first line is `+++ ' + uri + '` (replace <title> with an ASCII lowercase slug). The next line starts the markup. This is your only HTML destination. Use Edit on that URI for corrections.\n' +
'- FlowChat opens this exact source privately with the conversation renderer during publication. Write/Edit returns the render result. Repair reported issues on the same URI, then use the returned reference. All preview instructions in the guides, including “open it before replying” for maps, are fulfilled by this host check. Do not create scratch HTML, launch a server, open a browser, request screenshots, or produce tool-by-tool progress messages for verification.\n' +
'- For a city or regional marker map, publish the bundled offline atlas inline first. Write a heading and `<script type="application/json" data-visual-atlas="1">{"places":[{"name":"京都","country":"JP"},{"name":"大阪","country":"JP"}]}</script>` using the requested city names. Support for worldwide land, coasts, lakes, rivers, province boundaries, city coordinates, labels and projection is built in. It produces a complete standalone SVG in the saved reply. No downloads, npm, scratch scripts or local previews are necessary. Follow the adapted maps guide for other map types. For a moving simulation put data-visual-motion on the moving element or canvas.\n\n')
'- Choose a day calendar or timeline for a time-based itinerary. For today or tonight, verify the destination time and offer start times still available. An adjustable arrival time can shift its events and return time together. Choose a map for spatial questions. When both matter, combine them. For a city or regional marker map, publish the bundled offline atlas inline first. Write a heading and `<script type="application/json" data-visual-atlas="1">{"places":[{"name":"京都","country":"JP"},{"name":"大阪","country":"JP"}]}</script>` using the requested city names. Support for worldwide land, coasts, lakes, rivers, province boundaries, city coordinates, labels and projection is built in. It produces a complete standalone SVG in the saved reply. No downloads, npm, scratch scripts or local previews are necessary. ' + streetMaps + ' Follow the adapted maps guide for other map types. For a moving simulation put data-visual-motion on the moving element or canvas.\n\n')
.replace(/- Check that JavaScript has no undefined identifiers[^\n]+/,
'- Check selectors, JavaScript identifiers and the primary interaction. Use the FlowChat publication result for the render check.')
.replace(
Expand Down Expand Up @@ -115,7 +150,7 @@ for (const entry of Object.keys(manifest.files)) {
? flowChatEntry(await readFile(path.join(packagedGuides, name), 'utf8'), '../upstream/skills/visualize')
: name === 'maps.md'
? (await readFile(path.join(target, entry), 'utf8'))
.replace('# Maps', '# Maps\n\nFor city, regional and multi-city marker maps use the packaged regional atlas. Write the diagram once to the FlowChat URI with a heading and `<script type="application/json" data-visual-atlas="1">{"places":[{"name":"京都","country":"JP"},{"name":"大阪","country":"JP"}]}</script>`. A place takes its local name and optional ISO-2 country, first-level region, or sourced coordinates [longitude, latitude]. Optional bounds are [west,south,east,north]. Use this same path for any covered region. It already supplies real coastlines, rivers, lakes and boundaries. Use custom geometry only for more local detail or a layer this atlas does not contain.')
.replace('# Maps', '# Maps\n\nFor city, regional and multi-city marker maps use the packaged regional atlas. Write the diagram once to the FlowChat URI with a heading and `<script type="application/json" data-visual-atlas="1">{"places":[{"name":"京都","country":"JP"},{"name":"大阪","country":"JP"}]}</script>`. A place takes its local name and optional ISO-2 country, first-level region, or sourced coordinates [longitude, latitude]. Optional bounds are [west,south,east,north]. Use this same path for any covered region. It already supplies real coastlines, rivers, lakes and boundaries. ' + streetMaps + ' For a custom projected map mark the SVG with data-visual-map and data-map-source (source URL), and its real basemap geometry with data-visual-region.')
.replace('Open it before replying and fix', 'The background Write/Edit check opens the final fragment before the reply. Use its diagnostics to fix')
: await readFile(path.join(target, entry), 'utf8');
await syncGuide(path.join(owner, 'authoring', name), expected);
Expand Down
Loading
Loading