Can Laguna S 2.1 write working Godot 4 code? 9 of 15 on the first try
By Eduardo Orellana · Updated
Laguna S 2.1 passed 9 of 15 Godot 4 tasks on the first try and 12 of 15 after one fix with the test's output, each run headless on Godot 4.7.2. It missed: move a characterbody2d with an input action, save and load a dictionary as json, health with signals, a path on a grid around walls, build and play an animation in code, wait for a signal or a timeout.
AppImage, x86_64. Ubuntu 22.04, Debian 12, Fedora 39 or newer.
Intel or AMD Chromebook, with Linux turned on.
Every model measured on the same Godot 4 tasks, each run headless on Godot 4.7.2 at a fixed 60 frames per second.
Measured by Flockbay: the same tasks, the same checks, one fix allowed with the test output.
| Model | First try | After one fix | Agent | Run on | Read from |
|---|---|---|---|---|---|
| Claude Opus 5.5 | 15 of 15 | 15 of 15 | Claude Code | 2026-10-06 | Claude Opus 5.5 results |
| Claude Sonnet 5.5 | 15 of 15 | 15 of 15 | Claude Code | 2026-10-06 | Claude Sonnet 5.5 results |
| GPT-6.1 Sol | 15 of 15 | 15 of 15 | Codex | 2026-10-06 | GPT-6.1 Sol results |
| DeepSeek V4.1 Flash | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | DeepSeek V4.1 Flash results |
| Ember-1 | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Ember-1 results |
| Gemini 3.1 Pro Preview | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Gemini 3.1 Pro Preview results |
| Muse Wonder | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Muse Wonder results |
| Kimi K3 | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Kimi K3 results |
| GPT-5.6 Luna | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | GPT-5.6 Luna results |
| GPT-6 Luna | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | GPT-6 Luna results |
| Hy4 preview | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Hy4 preview results |
| Pareto 26.10 | 15 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Pareto 26.10 results |
| Claude Haiku 5.5 | 14 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | Claude Haiku 5.5 results |
| MiMo-V2.6-Pro-UltraSpeed | 14 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | MiMo-V2.6-Pro-UltraSpeed results |
| GLM 5.2 | 14 of 15 | 15 of 15 | Hosted, by API | 2026-10-18 | GLM 5.2 results |
| DeepSeek V4 Pro 0813 | 14 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | DeepSeek V4 Pro 0813 results |
| GPT-5.4 | 14 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | GPT-5.4 results |
| MiniMax M3 | 13 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | MiniMax M3 results |
| Qwen3.7 Max | 13 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | Qwen3.7 Max results |
| Fugu Max | 13 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | Fugu Max results |
| Beam 1.0 501B A23B | 12 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | Beam 1.0 501B A23B results |
| Mistral Large 4 | 12 of 15 | 13 of 15 | Hosted, by API | 2026-10-18 | Mistral Large 4 results |
| Claude Haiku 4.5 | 11 of 15 | 14 of 15 | Claude Code | 2026-10-06 | Claude Haiku 4.5 results |
| Nemotron 3.5 Super VL | 11 of 15 | 14 of 15 | Hosted, by API | 2026-10-18 | Nemotron 3.5 Super VL results |
| LongCat 2.0 | 11 of 15 | 13 of 15 | Hosted, by API | 2026-10-18 | LongCat 2.0 results |
| Grok 4.3 | 11 of 15 | 13 of 15 | Hosted, by API | 2026-10-18 | Grok 4.3 results |
| Laguna S 2.1 | 9 of 15 | 12 of 15 | Hosted, by API | 2026-10-18 | Laguna S 2.1 results |
| Nova 2 Lite | 2 of 15 | 4 of 15 | Hosted, by API | 2026-10-18 | Nova 2 Lite results |
How the test works
Each task asks for one GDScript file with a stated API, in a fresh Godot project. Laguna S 2.1 was asked through its API, the way Flockbay reaches its hosted models: the task in one request, no tools, and the whole file back in a code block. A check scene then loaded the file and drove it on Godot 4.7.2, headless, at a fixed 60 frames per second: it pressed input actions, stepped physics, waited on timers and signals, and compared the results.
When a task failed, the model got the check's output once and could fix its file. Every task touches an API that changed between Godot 3 and Godot 4, or was added since: move_and_slide and is_on_floor, FileAccess and JSON, signals, tweens, Timer nodes, PackedScene, TileMapLayer, ray queries, AStarGrid2D, AnimationLibrary, and lambdas, which capture local variables by value.
The checks themselves are tested first: each passes on a reference answer and fails on an empty script. Run on 2026-10-18.
Laguna S 2.1 in Flockbay
Laguna S 2.1 is a hosted model in the Flockbay app's model menu, on the Hobby plan and above. It uses the plan's hosted usage at about 0.8 times the rate of Auto, so it lasts longer.
It takes text only, so Flockbay describes the game to it rather than showing screenshots, and it has an effort setting in the composer, from quick answers to longer reasoning.
Move a CharacterBody2D with an input action: pass after one fix
The task: Write res://player.gd for Godot 4: a script for a CharacterBody2D that moves right at 200 pixels per second while the input action "ui_right" is held, and stands still otherwise.
Failed on the first try (BENCH FAIL: after 0.5 s holding ui_right x is 0.0, expected about 100), passed after one fix.
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
BENCH FAIL: after 0.5 s holding ui_right x is 0.0, expected about 100Fall with gravity and land on the floor: pass
The task: Write res://ball.gd for Godot 4: a script for a CharacterBody2D that falls with a gravity of 980 pixels per second squared and stops when it lands on the floor. The scene gives it its collision shape.
Passed on the first try.
extends CharacterBody2D
# Gravity in pixels per second squared
const GRAVITY := 980.0
func _physics_process(delta: float) -> void:
# Apply gravity when not on the floor
if not is_on_floor():
velocity.y += GRAVITY * delta
else:
# Stop vertical movement when landed on the floor
velocity.y = 0.0
# Move the character and handle collisions
move_and_slide()
A score label: pass
The task: Write res://score_label.gd for Godot 4: a script for a Label that shows "Score: 0" when it enters the scene, and has a method add(points: int) that adds to the score and updates the text to "Score: <total>".
Passed on the first try.
extends Label
var score: int = 0
func _ready() -> void:
text = "Score: %d" % score
func add(points: int) -> void:
score += points
text = "Score: %d" % score
Save and load a dictionary as JSON: fail
The task: Write res://save_game.gd for Godot 4: a script extending RefCounted with two static functions: save_data(data: Dictionary, path: String) -> void writes the dictionary to the file as JSON, and load_data(path: String) -> Dictionary reads it back.
Failed on the first try (SCRIPT ERROR: Parse Error: Cannot call non-static function "parse()" on the class "JSON" directly. Make an instance instead.) and after one fix (SCRIPT ERROR: Parse Error: Static function "print()" not found in base "GDScriptNativeClass".).
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
SCRIPT ERROR: Parse Error: Cannot call non-static function "parse()" on the class "JSON" directly. Make an instance instead.
at: GDScript::reload (res://save_game.gd:24)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Parse Error: Cannot get property from enum value.
at: GDScript::reload (res://save_game.gd:25)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Parse Error: Cannot get property from enum value.
at: GDScript::reload (res://save_game.gd:26)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Parse Error: Cannot get property from enum value.
at: GDScript::reload (res://save_game.gd:28)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
ERROR: Failed to load script "res://save_game.gd" with error "Parse error".
at: load (modules/gdscript/gdscript_resource_format.cpp:46)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Invalid call. Nonexistent function 'save_data' in base 'GDScript'.
at: _ready (res://bench_check.gd:9)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:9)Health with signals: pass after one fix
The task: Write res://health.gd for Godot 4: a script extending Node with health starting at 100, a signal health_changed(old_value: int, new_value: int), a signal died, and a method take_damage(amount: int) that lowers health without going below 0, emits health_changed when health changes, and emits died once, when health reaches 0.
Failed on the first try (SCRIPT ERROR: Parse Error: The variable type is being inferred from a Variant value, so it will be typed as Variant. (Warning treated as error.)), passed after one fix.
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
SCRIPT ERROR: Parse Error: The variable type is being inferred from a Variant value, so it will be typed as Variant. (Warning treated as error.)
at: GDScript::reload (res://health.gd:27)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:7)
SCRIPT ERROR: Parse Error: The variable type is being inferred from a Variant value, so it will be typed as Variant. (Warning treated as error.)
at: GDScript::reload (res://health.gd:38)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:7)
ERROR: Failed to load script "res://health.gd" with error "Parse error".
at: load (modules/gdscript/gdscript_resource_format.cpp:46)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:7)
SCRIPT ERROR: Invalid access to property or key 'health_changed' on a base object of type 'Node'.
at: _ready (res://bench_check.gd:14)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:14)Fade out with a tween: pass
The task: Write res://fader.gd for Godot 4: a script extending Node2D with a method fade_out(duration: float) that fades the node's modulate alpha from 1 to 0 over duration seconds with a tween, and a signal faded emitted when the fade ends.
Passed on the first try.
extends Node2D
signal faded
func fade_out(duration: float) -> void:
var tween := create_tween()
tween.tween_property(self, "modulate:a", 0.0, duration)
await tween.finished
emit_signal("faded")
Spawn on a timer, then stop: pass
The task: Write res://spawner.gd for Godot 4: a script extending Node that, once in the scene, adds a new child Node2D every 0.25 seconds, using a Timer, and stops after 4 children. It keeps the number spawned in a variable spawned_count.
Passed on the first try.
extends Node
var spawned_count: int = 0
var timer: Timer
func _ready() -> void:
timer = Timer.new()
timer.wait_time = 0.25
timer.autostart = true
timer.one_shot = false
add_child(timer)
timer.connect("timeout", Callable(self, "_on_timer_timeout"))
func _on_timer_timeout() -> void:
if spawned_count < 4:
var child = Node2D.new()
add_child(child)
spawned_count += 1
else:
timer.stop()
A door with states: pass
The task: Write res://door.gd for Godot 4: a script extending Node for a door with the states CLOSED, OPENING and OPEN. It starts CLOSED. A method interact() starts opening a closed door; 0.5 seconds of game time later it is OPEN. A method get_state_name() -> String returns "CLOSED", "OPENING" or "OPEN".
Passed on the first try.
# door.gd
# A simple door with states: CLOSED, OPENING, OPEN.
extends Node
# Door states.
const CLOSED := 0
const OPENING := 1
const OPEN := 2
# Initial state.
var state := CLOSED
# Timer handle for delayed opening.
var _open_timer : SceneTreeTimer = null
# Returns the current state as a readable name.
func get_state_name() -> String:
match state:
CLOSED:
return "CLOSED"
OPENING:
return "OPENING"
OPEN:
return "OPEN"
_:
return "UNKNOWN"
# Starts opening the door if it is currently closed.
func interact() -> void:
if state == CLOSED:
state = OPENING
# Wait 0.5 seconds of game time, then open.
_open_timer = get_tree().create_timer(0.5)
_open_timer.timeout.connect(_on_open_timeout)
# Called when the open delay finishes.
func _on_open_timeout() -> void:
state = OPEN
_open_timer = null
An inventory with counts: pass
The task: Write res://inventory.gd for Godot 4: a script extending RefCounted with add_item(item_name: String, count: int), remove_item(item_name: String, count: int) -> bool that removes only when there are enough and returns whether it did, and get_count(item_name: String) -> int that is 0 for an item never added.
Passed on the first try.
# res://inventory.gd
# Inventory manager that tracks item counts by name.
extends RefCounted
## The internal dictionary that maps item names to their counts.
var _items: Dictionary = {}
## Adds [count] of [item_name] to the inventory.
func add_item(item_name: String, count: int) -> void:
if count <= 0:
return
if _items.has(item_name):
_items[item_name] += count
else:
_items[item_name] = count
## Attempts to remove [count] of [item_name] from the inventory.
## Returns true only if the item existed in sufficient quantity
## and was successfully removed.
func remove_item(item_name: String, count: int) -> bool:
if count <= 0:
return false
if not _items.has(item_name):
return false
if _items[item_name] < count:
return false
_items[item_name] -= count
# Clean up the entry so it doesn't linger at zero.
if _items[item_name] <= 0:
_items.erase(item_name)
return true
## Returns the current count of [item_name] in the inventory.
## Returns 0 if the item has never been added or has been fully removed.
func get_count(item_name: String) -> int:
if _items.has(item_name):
return _items[item_name]
return 0
Load a level scene from a path: pass
The task: Write res://level_loader.gd for Godot 4: a script extending Node with a method load_level(path: String) -> Node that loads the scene file at path, creates an instance of it, adds it as a child, and returns it.
Passed on the first try.
extends Node
func load_level(path: String) -> Node:
var scene: PackedScene = load(path)
var instance: Node = scene.instantiate()
add_child(instance)
return instance
Paint cells on a TileMapLayer: pass
The task: Write res://grid_painter.gd for Godot 4.7: a script extending TileMapLayer with paint_row(y: int, from_x: int, to_x: int) that sets the cells from (from_x, y) to (to_x, y) inclusive to tile source 0, atlas coordinates (0, 0), and is_painted(cell: Vector2i) -> bool that says whether a cell has a tile.
Passed on the first try.
extends TileMapLayer
func paint_row(y: int, from_x: int, to_x: int) -> void:
for x in range(from_x, to_x + 1):
set_cell(Vector2i(x, y), 0, Vector2i(0, 0))
func is_painted(cell: Vector2i) -> bool:
return get_cell_source_id(cell) != -1
Cast a physics ray: pass
The task: Write res://line_of_sight.gd for Godot 4: a script extending Node2D with first_hit(from: Vector2, to: Vector2) -> Vector2 that casts a physics ray through the 2D world from `from` to `to` and returns the point where it first hits a collider, or `to` when it hits nothing. It is called from physics processing.
Passed on the first try.
extends Node2D
func first_hit(from: Vector2, to: Vector2) -> Vector2:
var space_state = get_world_2d().direct_space_state
var query = PhysicsRayQueryParameters2D.create(from, to)
var result = space_state.intersect_ray(query)
if result:
return result.position
return to
A path on a grid around walls: pass after one fix
The task: Write res://path_finder.gd for Godot 4: a script extending RefCounted with find_path(size: Vector2i, walls: Array[Vector2i], from: Vector2i, to: Vector2i) -> Array[Vector2i] that returns the shortest path of grid cells from `from` to `to`, both included, on a grid of `size` cells, never entering a wall cell and moving only up, down, left or right.
Failed on the first try (SCRIPT ERROR: Parse Error: Invalid argument for "_is_valid_cell()" function: argument 3 should be "Dictionary" but is "Array[Vector2i]".), passed after one fix.
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
SCRIPT ERROR: Parse Error: Invalid argument for "_is_valid_cell()" function: argument 3 should be "Dictionary" but is "Array[Vector2i]".
at: GDScript::reload (res://path_finder.gd:17)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Parse Error: Invalid argument for "_is_valid_cell()" function: argument 3 should be "Dictionary" but is "Array[Vector2i]".
at: GDScript::reload (res://path_finder.gd:17)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Parse Error: The variable type is being inferred from a Variant value, so it will be typed as Variant. (Warning treated as error.)
at: GDScript::reload (res://path_finder.gd:33)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
ERROR: Failed to load script "res://path_finder.gd" with error "Parse error".
at: load (modules/gdscript/gdscript_resource_format.cpp:46)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Invalid call. Nonexistent function 'new' in base 'GDScript'.
at: _ready (res://bench_check.gd:9)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:9)Build and play an animation in code: fail
The task: Write res://blinker.gd for Godot 4: a script extending Node2D that, when it enters the scene, creates an AnimationPlayer child and a looping animation named "blink", 1 second long, that keeps the node visible for the first half of each second and hidden for the second half, and plays it.
Failed on the first try (BENCH FAIL: visible at 0.25, 0.75, 1.25 and 1.75 s was [true, true, true, true], expected [true, false, true, false]) and after one fix (BENCH FAIL: visible at 0.25, 0.75, 1.25 and 1.75 s was [true, true, true, true], expected [true, false, true, false]).
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
SCRIPT ERROR: Invalid call. Nonexistent function 'track_set_key_count' in base 'Animation'.
at: _ready (res://blinker.gd:14)
GDScript backtrace (most recent call first):
[0] _ready (res://blinker.gd:14)
[1] _ready (res://bench_check.gd:10)
BENCH FAIL: visible at 0.25, 0.75, 1.25 and 1.75 s was [true, true, true, true], expected [true, false, true, false]Wait for a signal or a timeout: fail
The task: Write res://waiter.gd for Godot 4: a script extending Node with wait_for(sig: Signal, seconds: float) -> bool, a coroutine that waits until the signal is emitted or `seconds` of game time pass, whichever comes first, and returns true if the signal came first. The signal has no arguments.
Failed on the first try (SCRIPT ERROR: Parse Error: Expected closing ")" after grouping expression.) and after one fix (SCRIPT ERROR: Parse Error: "yield" was removed in Godot 4. Use "await" instead.).
What Godot 4.7.2 printed on the first try:
Godot Engine v4.7.2.stable.mono.official.ed1daf0bf - https://godotengine.org
SCRIPT ERROR: Parse Error: Expected closing ")" after grouping expression.
at: GDScript::reload (res://waiter.gd:33)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
ERROR: Failed to load script "res://waiter.gd" with error "Parse error".
at: load (modules/gdscript/gdscript_resource_format.cpp:46)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:4)
SCRIPT ERROR: Invalid call. Nonexistent function 'wait_for' in base 'Node'.
at: _ready (res://bench_check.gd:13)
GDScript backtrace (most recent call first):
[0] _ready (res://bench_check.gd:13)Questions
- Is Laguna S 2.1 good at Godot?
- On these 15 tasks it passed 9 on the first try and 12 after one fix. Each task is small and checked by running it, so the score says whether its Godot 4 code works, not how well it designs a game.
Related
- Grok 4.3 on the same tasks
- LongCat 2.0 on the same tasks
- Mistral Large 4 on the same tasks
- Beam 1.0 501B A23B on the same tasks
- Claude Haiku 4.5 on the same tasks
- DeepSeek V4 Pro 0813 on the same tasks
- Fugu Max on the same tasks
- GPT-5.4 on the same tasks
- Every verified Godot 4 answer
- Can Nemotron 3.5 Super VL write working Godot 4 code? 11 of 15 on the first try
- Can Qwen3.7 Max write working Godot 4 code? 13 of 15 on the first try
- Can MiniMax M3 write working Godot 4 code? 13 of 15 on the first try
- Can Nova 2 Lite write working Godot 4 code? 2 of 15 on the first try
- Can GLM 5.2 write working Godot 4 code? 14 of 15 on the first try
Flockbay runs your own agent, Claude or ChatGPT on the official Godot 4.7.2, and gives it Godot's errors the way this test does.
Download Flockbay free