拾穗数据工作室SQL 模型评测台

完成但有错误 · 2026年8月30日 00:25

运行 #11 评测报告

Sol 与 Gemini,12 道题。综合得分为规则加权分,不是正确率。

比较条件存在差异:adapter_kind、cli_version。阅读模型间差距时必须保留这一前提。

失败记录:13 个失败案例已按固定规则计分,详情见逐题结果。

逐题结果原始报告 JSON ↗事件 JSONL ↗
gpt-5.6-sol

Sol

综合得分 / 100
89.79
gemini-2.5-pro

Gemini

综合得分 / 100
0.00
12题目数量
1每题尝试次数
13失败案例

样本边界:结论仅适用于此题库哈希、模型版本、适配器和参数快照。跨版本、跨运行稳定性需要独立复测。

分差最大的题目:基础筛选与排序

分差按每个模型在该题全部计划尝试的平均得分计算。

结果预测(可选)

选择你认为本题全部作答结果正确的模型。选择仅保存在本机,不上传、不计票;也可直接展开结果对比。这不是盲测。

选择后显示根据评分分项派生的结果比较。

查看结果对比

模型 A · Sol

100.00全部 1 次作答均分 · 结果正确(页面派生)

A1 · 查询 2026-01-01 起的已完成订单及客户名称,按订单总额降序、订单 ID 升序排列。

查看 A1 的执行与评分证据 →

模型 B · Gemini

0.00全部 1 次作答均分 · 结果正确性未知

A1 · Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

查看 A1 的执行与评分证据 →

历史结果正确要求全部计划尝试的执行、行结果、列名、列数与排序分项均通过;AST 不参与。任一作答缺少证据即显示未知。

得分与资源消耗

新 efficiency-v2 按业务结果正确题数归一;旧 efficiency-v1 保留综合得分 ÷ 100 的折算口径,两种指标不能混排。记录不完整时不估算单位正确题消耗。

指标SolGemini
综合得分
固定规则加权 / 100
89.790.00
Token / 归一单位19,773每得分折算题 · 11/12 题有记录每得分折算题 · 未记录
费用 / 归一单位
基于冻结价格;缺失不估
不可估算每得分折算题 · unavailable不可估算每得分折算题 · unavailable
生成耗时中位数
模型生成阶段
P95 —P95 —

分项得分

分项为加权得分,不是业务结果正确率。

题目维度SolGemini
基础查询100.000.00
复杂管道96.250.00
子查询100.000.00
窗口函数71.250.00
聚合分析100.000.00
连接语义100.000.00

逐题结果

得分为全部计划尝试的均分,失败计入分母,未完成显示未知。展开可查看每次作答。

基础筛选与排序basic_filter_sort100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

查询 2026-01-01 起的已完成订单及客户名称,按订单总额降序、订单 ID 升序排列。

结果正确(页面派生)

SELECT
  o.order_id,
  c.customer_name,
  o.order_date,
  o.total_amount
FROM fact_orders AS o
JOIN dim_customers AS c
  ON o.customer_id = c.customer_id
WHERE
  o.status = 'completed' AND o.order_date >= CAST('2026-01-01' AS DATE)
ORDER BY
  o.total_amount DESC,
  o.order_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
多表连接品类收入multi_join_category_revenue100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

按商品品类汇总已完成订单的去重订单数和订单行净销售额,并按净销售额降序、品类升序排列。

结果正确(页面派生)

SELECT
  p.category,
  COUNT(DISTINCT o.order_id) AS order_count,
  SUM(oi.quantity * oi.unit_price - oi.discount_amount) AS net_revenue
FROM fact_orders AS o
JOIN fact_order_items AS oi
  ON oi.order_id = o.order_id
JOIN dim_products AS p
  ON p.product_id = oi.product_id
WHERE
  o.status = 'completed'
GROUP BY
  p.category
ORDER BY
  net_revenue DESC,
  p.category ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
保留零订单客户left_join_zero_orders100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

列出所有客户及其已完成订单数,通过 LEFT JOIN 保留零订单客户,并按完成订单数降序、客户 ID 升序排列。

结果正确(页面派生)

SELECT
  c.customer_id,
  c.customer_name,
  COUNT(DISTINCT o.order_id) AS completed_order_count
FROM dim_customers AS c
LEFT JOIN fact_orders AS o
  ON c.customer_id = o.customer_id AND o.status = 'completed'
GROUP BY
  c.customer_id,
  c.customer_name
ORDER BY
  completed_order_count DESC,
  c.customer_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
相关子查询分群均值correlated_segment_average100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

返回金额高于其客户分群已完成订单平均金额的已完成订单,并按订单金额降序、订单 ID 升序排列。

结果正确(页面派生)

SELECT
  o.order_id,
  o.customer_id,
  c.segment,
  o.total_amount,
  (
    SELECT
      AVG(o2.total_amount)
    FROM fact_orders AS o2
    JOIN dim_customers AS c2
      ON c2.customer_id = o2.customer_id
    WHERE
      o2.status = 'completed' AND c2.segment = c.segment
  ) AS segment_avg_amount
FROM fact_orders AS o
JOIN dim_customers AS c
  ON c.customer_id = o.customer_id
WHERE
  o.status = 'completed'
  AND o.total_amount > (
    SELECT
      AVG(o3.total_amount)
    FROM fact_orders AS o3
    JOIN dim_customers AS c3
      ON c3.customer_id = o3.customer_id
    WHERE
      o3.status = 'completed' AND c3.segment = c.segment
  )
ORDER BY
  o.total_amount DESC,
  o.order_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
从未完成订单客户not_exists_customers100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

返回从未有已完成订单的客户,并按 customer_id 升序排列。

结果正确(页面派生)

SELECT
  c.customer_id,
  c.customer_name
FROM dim_customers AS c
WHERE
  NOT EXISTS(
    SELECT
      1
    FROM fact_orders AS o
    WHERE
      o.customer_id = c.customer_id AND o.status = 'completed'
  )
ORDER BY
  c.customer_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
每品类收入前三商品top3_products_per_category100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

汇总完成订单中各商品的净销售额,并使用 ROW_NUMBER 选出每个品类净销售额前三的商品;同额时按 product_id 升序排名。

结果正确(页面派生)

WITH product_revenue AS (
  SELECT
    p.category,
    p.product_id,
    p.product_name,
    SUM(oi.quantity * oi.unit_price - oi.discount_amount) AS net_revenue
  FROM fact_order_items AS oi
  JOIN fact_orders AS o
    ON oi.order_id = o.order_id
  JOIN dim_products AS p
    ON oi.product_id = p.product_id
  WHERE
    o.status = 'completed'
  GROUP BY
    p.category,
    p.product_id,
    p.product_name
), ranked_products AS (
  SELECT
    category,
    product_id,
    product_name,
    net_revenue,
    ROW_NUMBER() OVER (PARTITION BY category ORDER BY net_revenue DESC, product_id ASC) AS rank_no
  FROM product_revenue
)
SELECT
  category,
  product_id,
  product_name,
  net_revenue,
  rank_no
FROM ranked_products
WHERE
  rank_no <= 3
ORDER BY
  category ASC,
  rank_no ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
月收入与累计收入monthly_running_revenue100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

按月汇总 2025 年已完成订单的净销售收入,并使用按月份排序的 SUM 窗口计算累计收入。

结果正确(页面派生)

WITH monthly AS (
  SELECT
    STRFTIME(o.order_date, '%Y-%m') AS month,
    SUM(oi.quantity * oi.unit_price - oi.discount_amount) AS monthly_revenue
  FROM fact_orders AS o
  JOIN fact_order_items AS oi
    ON oi.order_id = o.order_id
  WHERE
    o.status = 'completed'
    AND o.order_date >= CAST('2025-01-01' AS DATE)
    AND o.order_date < CAST('2026-01-01' AS DATE)
  GROUP BY
    1
)
SELECT
  month,
  monthly_revenue,
  SUM(monthly_revenue) OVER (ORDER BY month ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS cumulative_revenue
FROM monthly
ORDER BY
  month ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
月收入环比增长monthly_mom_growth0.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

FROM 不允许表函数

结果正确性未知

查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
高于平均累计消费客户above_average_customer_spend100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

汇总每位有已完成订单客户的累计消费,以这些客户的平均累计消费为基准,返回高于平均值的客户。

结果正确(页面派生)

SELECT
  customer_id,
  customer_name,
  total_spend
FROM (
  SELECT
    customer_id,
    customer_name,
    total_spend,
    AVG(total_spend) OVER () AS avg_total_spend
  FROM (
    SELECT
      c.customer_id,
      c.customer_name,
      SUM(o.total_amount) AS total_spend
    FROM fact_orders AS o
    JOIN dim_customers AS c
      ON o.customer_id = c.customer_id
    WHERE
      o.status = 'completed'
    GROUP BY
      c.customer_id,
      c.customer_name
  ) AS customer_spend
) AS spend_with_average
WHERE
  total_spend > avg_total_spend
ORDER BY
  total_spend DESC,
  customer_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
渠道支付状态金额payment_status_by_channel100.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 100.00

按渠道类型汇总成功支付、退款状态和失败支付金额,并按渠道类型升序排列。

结果正确(页面派生)

SELECT
  c.channel_type,
  SUM(CASE WHEN p.status = 'paid' THEN p.amount ELSE 0 END) AS paid_amount,
  SUM(CASE WHEN p.status = 'refunded' THEN p.amount ELSE 0 END) AS refunded_amount,
  SUM(CASE WHEN p.status = 'failed' THEN p.amount ELSE 0 END) AS failed_amount
FROM fact_payments AS p
JOIN fact_orders AS o
  ON p.order_id = o.order_id
JOIN dim_channels AS c
  ON o.channel_id = c.channel_id
GROUP BY
  c.channel_type
ORDER BY
  c.channel_type ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
品类退货率category_return_rate92.50 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 92.50

分别按品类聚合完成订单的售出数量与退货数量,再计算四舍五入至 4 位的退货率,并按退货率降序、品类升序排列。

结果正确(页面派生)

WITH sold AS (
  SELECT
    p.category,
    SUM(oi.quantity) AS sold_qty
  FROM fact_order_items AS oi
  JOIN fact_orders AS o
    ON o.order_id = oi.order_id
  JOIN dim_products AS p
    ON p.product_id = oi.product_id
  WHERE
    o.status = 'completed'
  GROUP BY
    p.category
), returned AS (
  SELECT
    p.category,
    SUM(r.return_qty) AS returned_qty
  FROM fact_returns AS r
  JOIN fact_order_items AS oi
    ON oi.order_id = r.order_id AND oi.line_no = r.line_no
  JOIN fact_orders AS o
    ON o.order_id = oi.order_id
  JOIN dim_products AS p
    ON p.product_id = oi.product_id
  WHERE
    o.status = 'completed'
  GROUP BY
    p.category
), category_rates AS (
  SELECT
    s.category,
    s.sold_qty,
    COALESCE(r.returned_qty, 0) AS returned_qty,
    ROUND(CAST(COALESCE(r.returned_qty, 0) AS DECIMAL(18, 3)) / NULLIF(s.sold_qty, 0), 4) AS return_rate
  FROM sold AS s
  LEFT JOIN returned AS r
    ON r.category = s.category
)
SELECT
  category,
  sold_qty,
  returned_qty,
  return_rate
FROM category_rates
ORDER BY
  return_rate DESC,
  category ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据
每单最新成功支付latest_successful_payment85.00 / 0.00

Sol

全部 1 次计划作答 · 结果正确(页面派生)

A1 · 完成 · 85.00

筛选成功支付,并使用 ROW_NUMBER 为每个订单按支付时间和 payment_id 降序排名,返回最新一笔。

结果正确(页面派生)

SELECT
  order_id,
  payment_id,
  paid_at,
  amount
FROM (
  SELECT
    order_id,
    payment_id,
    paid_at,
    amount,
    ROW_NUMBER() OVER (PARTITION BY order_id ORDER BY paid_at DESC, payment_id DESC) AS rn
  FROM fact_payments
  WHERE
    status = 'paid'
) AS ranked_payments
WHERE
  rn = 1
ORDER BY
  order_id ASC
查看 A1 完整证据

Gemini

全部 1 次计划作答 · 结果正确性未知

A1 · 失败 · 0.00

Gemini 退出码 41: Please set an Auth method in your $PROJECT_ROOT/var/cli-homes/gemini/3/.gemini/settings.json or specify one of the following environment variables before running: GEMINI_API_KEY, GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_GENAI_USE_GCA

结果正确性未知

查看 A1 完整证据