LLM Selection

Key takeaway

Leading flagship models currently available on the market can all build the core of a NocoBase application.

Models differ in the completeness of their initial output, build time, and number of issues. Choose one based on the model services already available to you, network conditions in your region, cost, and your team's preferences.

This evaluation used a standardized CRM requirement set (a sales opportunity and customer follow-up system) to validate the applications built by different models:

Evaluation dimensions	Standardized evaluation items
14	61

Evaluation dimensions

The evaluation covers NocoBase's core capabilities, configuration capabilities, and foundational components. It also checks whether each model can understand requirements and carry out the corresponding build tasks.

Capability	Evaluation focus
Data modeling	Collections, field types, relationships, required and unique constraints, and default values
Pages and features	Navigation, lists, forms, details, search, filters, and dashboards
Business logic	Status transitions, business validation, calculation rules, and consistency of related data
Permissions and security	Roles, menu permissions, action permissions, data scopes, and field permissions
Workflow automation	Triggers, nodes, conditional branches, notifications, data side effects, and failure retries
User experience	Information architecture, form experience, action feedback, and responsive layouts
Robustness	Invalid input, duplicate submissions, consistency on failure, data volume, and network recovery
Requirements coverage	Whether explicit requirements and core business paths are fully implemented
Reasonable extensions	Whether features proactively added by the model serve a clear business purpose
Scope control	Whether the result contains duplicate, unused, or out-of-scope business modules

Evaluation results

Evaluation dimension	GPT-5.6 Sol	DeepSeek-V4-Flash	Qwen3.8-Max	GPT-5.6 Luna
Data modeling	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Feature completion	✓ Pass	✓ Pass	✓ Pass	◐ Partial pass
Business logic	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Permissions and security	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Workflow automation	✓ Pass	✓ Pass	✓ Pass	✓ Pass
User experience	✓ Pass	✓ Pass	✓ Pass	◐ Partial pass
Robustness	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Requirements coverage	✓ Pass	✓ Pass	✓ Pass	◐ Partial pass
Reasonable extensions	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Scope control	✓ Pass	✓ Pass	✓ Pass	✓ Pass
Build speed	Relatively fast	Relatively fast	Slow	Fastest
Single-run quality score	90	91	90	77

Single-run quality score

The single-run quality score has a maximum of 100 points. One point is deducted for each bug found during the first complete acceptance check, providing an indication of the quality of the model's initial build. The model can resolve these issues through subsequent feedback and revisions.

Note on build time

Build time is affected by factors such as computer hardware performance, dependency installation and Build compilation, model service response speed, and network conditions.

Evaluation item details

The 61 standardized evaluation items are organized into three layers: 46 items for build result quality, 7 for requirements understanding and reasonable extensions, and 8 for build process efficiency. Every item uses consistent inspection methods and pass criteria.

Layer 1: Build result quality (46 items)

Evaluation dimension	Standardized evaluation items
Data modeling (8 items)	`DM-01` Whether all required collections are created `DM-02` Whether all required fields exist `DM-03` Whether field types are correct `DM-04` Whether one-to-one relationships can be created and used `DM-05` Whether one-to-many relationships can be created and used `DM-06` Whether many-to-many relationships can be created and used `DM-07` Whether required, unique, and default-value rules take effect `DM-08` Whether related data can be viewed and filtered
Feature completion (6 items)	`FC-01` Whether all required pages and navigation entries are present `FC-02` Whether records can be created, viewed, edited, and deleted `FC-03` Whether core user journeys can be completed end to end `FC-04` Whether key business actions are available `FC-05` Whether search, filtering, and sorting are available `FC-06` Whether dashboards contain the required content
Business logic (6 items)	`BL-01` Whether opportunity status transition rules are correct `BL-02` Whether business validation rules take effect `BL-03` Whether calculated fields and statistical definitions are correct `BL-04` Whether data is mapped correctly after lead conversion `BL-05` Whether updates to related records remain consistent `BL-06` Whether deletion and archiving rules are correct
Permissions and security (7 items)	`ACL-01` Whether all required roles are created `ACL-02` Whether test users and role assignments are correct `ACL-03` Whether page and menu access permissions are correct `ACL-04` Whether data operation permissions are correct `ACL-05` Whether record-level data scopes are correct `ACL-06` Whether field-level view and edit permissions are correct `ACL-07` Whether role changes and combined roles behave correctly
Workflow automation (7 items)	`WF-01` Whether all required workflows are created and enabled `WF-02` Whether workflow triggers are designed correctly `WF-03` Whether node order and data transfer are correct `WF-04` Whether conditions and branch results are correct `WF-05` Whether record read/write side effects are correct `WF-06` Whether notification recipients and content are correct `WF-07` Whether failure logs and retry behavior are traceable
User experience (7 items)	`UX-01` Whether navigation and information architecture are clear `UX-02` Whether list information and common actions are easy to use `UX-03` Whether form grouping, order, and guidance are clear `UX-04` Whether detail pages support understanding and follow-up actions `UX-05` Whether action feedback and status changes are clear `UX-06` Whether the application is usable at different screen widths `UX-07` Whether empty, loading, and error states are complete
Robustness (5 items)	`ROB-01` Whether invalid and boundary inputs are handled safely `ROB-02` Whether duplicate submissions cause duplicate side effects `ROB-03` Whether data remains consistent when execution fails `ROB-04` Whether the application remains usable with empty and large datasets `ROB-05` Whether the application can recover after a session or network interruption

Layer 2: Requirements understanding and reasonable extensions (7 items)

Evaluation dimension	Standardized evaluation items
Requirements coverage (3 items)	`COV-01` Whether all pages and actions requested in the prompt are implemented `COV-02` Whether all data, permissions, and workflows requested in the prompt are implemented `COV-03` Whether capabilities required by the main process but not individually specified in the prompt are present
Reasonable extensions (2 items)	`EXT-01` Whether proactively added fields, relationships, and rules are necessary `EXT-02` Whether proactively added pages, actions, and statistics serve a clear purpose
Scope control (2 items)	`SCOPE-01` Whether duplicate or unused features and configurations are generated `SCOPE-02` Whether business modules unrelated to the task scope are added

Layer 3: Build process efficiency (8 items)

Evaluation dimension	Standardized evaluation items
Time to first usable result (1 item)	`EFF-FIRST-01` Time required to reach the first usable result
Convergence efficiency (3 items)	`EFF-FINAL-01` Number of iterations required to reach final acceptance `EFF-FINAL-02` Total time required to reach the final state `EFF-FINAL-03` Tokens consumed to reach the final state
Human intervention (1 item)	`EFF-HUMAN-01` Number of human interventions during the evaluation
Repeatability (3 items)	`EFF-STABLE-01` Whether repeated runs of the same task produce consistent acceptance results `EFF-STABLE-02` Whether collections, relationships, roles, and workflows are consistent across three runs `EFF-STABLE-03` Whether variation in iterations and time remains controlled

Next steps

Build collaboratively with an AI Agent — Describe pages and interactions in natural language and iterate continuously with an AI Agent
AI Portal quick start — Create and run your first AI Portal
Data modeling — Create collections, fields, and relationships with natural language
Workflow management — Create, edit, enable, and diagnose workflows
Permission configuration — Manage roles, permission policies, user assignments, and risk assessments

#LLM Selection

#Evaluation dimensions

#Evaluation results

#Evaluation item details

#Layer 1: Build result quality (46 items)

#Layer 2: Requirements understanding and reasonable extensions (7 items)

#Layer 3: Build process efficiency (8 items)

#Next steps