Add Instanced challenges by gqvz · Pull Request #450 · sdslabs/beast

gqvz · 2026-02-05T19:29:21Z

This PR introduces instanced challenges: a new challenge type where each contestant gets their own isolated container environment, automatically cleaned up after expiration. It also replaces the old static port-list approach with a Redis-backed dynamic port allocation system.

gqvz · 2026-02-05T19:30:24Z

will fix (squash) commits soon

Copilot

Pull request overview

This pull request adds support for instanced challenges - a feature that allows each user to spawn their own isolated container instance for a challenge, rather than sharing a single deployed instance. This is particularly useful for challenges that require per-user isolation, such as pwn challenges or vulnerable web applications.

Changes:

Adds Redis as a caching layer for managing dynamic port allocation and instance metadata
Implements instance lifecycle management (spawn, extend, kill) with automatic expiration
Introduces new database fields (Instanced, InstanceExpiration) to the Challenge model
Adds comprehensive API endpoints for users and admins to manage instances
Removes static port allocation from challenge configs in favor of dynamic allocation from port ranges
Includes example challenges demonstrating both simple service and docker-compose based instancing

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
go.mod, go.sum	Adds redis client dependency and removes unused dependencies
core/config/config.go	Adds Redis config, instance config, and port range validation
core/config/challenge.go	Adds instanced metadata fields and simplifies port configuration
core/database/challenges.go	Adds Instanced and InstanceExpiration fields to Challenge model
core/cache/*.go	New package for Redis-based caching of instances and port allocations
core/manager/instance.go	New file implementing instance lifecycle management
core/manager/pipeline.go	Updates deployment to use dynamic port allocation
core/manager/health_check.go	Adds instance cleanup prober for expired instances
core/manager/challenge.go	Updates undeploy to kill active instances first
core/manager/utils.go	Updates port registration to use cache instead of static config
api/instance.go	New API handlers for instance management
api/router.go	Adds instance API routes
cmd/beast/*.go	Adds Redis initialization and cache management commands
utils/datatypes.go	Updates port mapping comment to reflect new usage
_examples/instanced-*	Example challenges demonstrating instancing feature

Comments suppressed due to low confidence (1)

core/manager/pipeline.go:387

The error check at line 382 is performed after registering ports (lines 375-380). If the container creation failed (err != nil), the ports should not be registered. The error check should be moved before port registration to avoid registering ports for a failed container creation. Additionally, if the container creation fails, the allocated ports should be freed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-06T11:58:54Z

+# DO NOT specify ports for instanced challenges!
+# Instead, use default_port to indicate which container port to expose


The comment "# DO NOT specify ports for instanced challenges!" conflicts with the validation logic. According to core/config/challenge.go line 308, either ports or default_port must be specified. The comment should be updated to clarify that you should use default_port instead of ports for instanced challenges, not that you should avoid specifying any port information.

Suggested change

# DO NOT specify ports for instanced challenges!

# Instead, use default_port to indicate which container port to expose

# Do not configure a ports list for instanced challenges.

# Instead, you must use default_port to indicate which container port to expose.

Copilot · 2026-02-06T11:58:55Z

+tags = ["web", "sql", "instanced"]
+maxAttemptLimit = 100
+instanced = true
+instance_expiration = 12


The instance_expiration value is set to 12 seconds, which is extremely short for a challenge instance. This seems like it's intended for testing purposes. For actual use, this should be a more reasonable duration (e.g., 600 seconds as mentioned in the README). Consider updating this to match the recommended value or adding a comment indicating this is a test value.

Suggested change

instance_expiration = 12

instance_expiration = 600

Copilot · 2026-02-06T11:58:58Z

+// Postgresql database for beast. The Db variable is the connection variable for the
+// database, which is not closed after creating a connection here and can


The comment says "Postgresql database" but this is for cache/Redis initialization. This comment was likely copied from the database package and should be updated to reflect that this is for Redis/cache initialization.

Suggested change

// Postgresql database for beast. The Db variable is the connection variable for the

// database, which is not closed after creating a connection here and can

// Redis cache for beast. The Cache variable is the client connection for the

// cache, which is not closed after creating a connection here and can

v1bh475u · 2026-03-05T17:02:30Z

+	projectName := fmt.Sprintf("beast-instance-%s-%s", coreUtils.EncodeID(challengeName), instanceID)
+	composeFile := filepath.Join(stagingDir, challengeName, config.Challenge.Env.DockerCompose)
+
+	if serverDeployed == core.LOCALHOST || serverDeployed == "" {


I don't understand the disproportion here. Why this function manually executing the command when the same thing can be done using the docker compose for localhost as well! I think it might need some refactoring but spilling the logic in this function seems very redundant.

v1bh475u · 2026-03-05T17:03:24Z

+	return core.LOCALHOST
+}
+
+func deployInstanceContainer(instanceID, challengeName string, hostPort uint32, imageID string, config *cfg.BeastChallengeConfig, serverDeployed string) (string, error) {


It seems to me that both the deploy functions and killing the instances should not be here but in the pkg/cr and pkg/remoteManager. Either that or the logic of deployment and everything should be in those files not in the functions mentioned here. Thoughts?

v1bh475u · 2026-03-05T17:06:50Z

+	return ttl, nil
+}
+
+func QueueInstanceForDeletion(instanceID string) error {


I am not convinced of the idea that using this queue and the probing logic is a good fit for our problem

what other clean up mechanisms can we use?

…word

…rd arguments

Cleanup: Remove unused handler. Fix: fix typo. Fix: fix port range. Refactor: refactor constant paths. Fix: fix variable management. Update: fix port range in example Fix: Add descriptive errors. Fix: fix error handling. Feat: Cleanup undeployment Sanity: add sanity init in cache helpers

…lve timestamps and optimized database queries

Copilot

Pull request overview

Copilot reviewed 48 out of 49 changed files in this pull request and generated 14 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-13T09:52:12Z

+var portRegex = regexp.MustCompile(`\$\{([^}]+)}`)
+
+func ExtractPortsFromCompose(composeFile string) ([]string, error) {
+	data, err := os.ReadFile(composeFile)
+	if err != nil {
+		return nil, fmt.Errorf("error while reading compose file: %w", err)
+	}
+	var raw Compose
+	err = yaml.Unmarshal(data, &raw)
+	if err != nil {
+		return nil, fmt.Errorf("error while parsing compose file: %s", err.Error())
+	}
+
+	portVariables := make([]string, 0)
+	seen := make(map[string]bool)
+	for _, service := range raw.Services {
+		for _, port := range service.Ports {
+			matches := portRegex.FindAllStringSubmatch(port, -1)
+
+			if len(matches) == 0 {
+				return nil, fmt.Errorf("port %s is not mapped using an env variable", port)
+			}


ExtractPortsFromCompose currently captures the full ${...} expression, so a mapping like ${INSTANCE_PORT:-8080}:80 will yield variable name INSTANCE_PORT:-8080, which is not a valid env var name and will break dynamic port injection. Also, returning an error for port mappings without ${...} will now fail existing non-instanced compose challenges (e.g., _examples/compose-type/docker-compose.yml uses "10020:80"). Consider parsing only the variable name portion (strip :-default / -default modifiers) and either supporting fixed mappings or gating this requirement to instanced/dynamic-port compose deployments with a clearer error message.

Copilot · 2026-04-13T09:52:12Z

+		server := cfg.Cfg.AvailableServers[serverDeployed]
+		if deploymentType == core.DEPLOYMENT_TYPES["docker_compose"] {
+			stagingDir := filepath.Join(core.BEAST_GLOBAL_DIR, core.BEAST_STAGING_DIR)
+			err := remoteManager.ComposePurgeRemote(challengeName, stagingDir, server)
+			if err != nil {
+				return fmt.Errorf("failed to stop compose on remote: %w", err)
+			}


killInstanceContainer uses ComposePurgeRemote(challengeName, ...), but instanced compose deployments use a per-instance project name (via projectName := utils.GetProjectName(projectBase) where projectBase is instance-...). This will purge beast-<challengeName> instead of beast-instance-..., leaving remote instance projects/containers orphaned and ports potentially leaked. Pass the instance project base/name through to the remote purge/down call (or add a remote purge helper that accepts an explicit project name).

Copilot · 2026-04-13T09:52:13Z

+func instanceToResponse(instance *cache.Instance) InstanceResponse {
+	ttl := time.Until(instance.ExpiresAt).Seconds()
+	if ttl < 0 {
+		ttl = 0
+	}
+
+	return InstanceResponse{
+		InstanceID:    instance.InstanceID,
+		ChallengeName: instance.ChallengeName,
+		Port:          instance.Port,
+		CreatedAt:     instance.CreatedAt,
+		ExpiresAt:     instance.ExpiresAt,
+		TTLSeconds:    int64(ttl),
+	}


instanceToResponse never populates HostedAddress, so API responses will always return an empty hosted_address even though clients need the server hostname/IP. Use instance.ServerDeployed (and map it to config.Cfg.AvailableServers[name].Host when applicable, similar to challengeInfoHandler) to set HostedAddress.

Copilot · 2026-04-13T09:52:13Z

+		Select("DISTINCT ON (user_challenges.user_id, user_challenges.challenge_id) user_challenges.user_id, users.username, user_challenges.created_at, challenges.points, false AS is_hint").
 		Joins("JOIN challenges ON user_challenges.challenge_id = challenges.id").
 		Joins("JOIN users ON user_challenges.user_id = users.id").
 		Where("user_challenges.user_id IN ? AND user_challenges.solved = ?", topUserId, true).


The user_challenges query uses DISTINCT ON (user_id, challenge_id) but no longer specifies an ORDER BY. With DISTINCT ON, PostgreSQL selects an arbitrary row unless ordering is explicit, which can produce incorrect/unstable solve timestamps and time-series scores. Add an ORDER BY user_id, challenge_id, created_at ASC (or DESC, depending on desired semantics) to deterministically pick the intended row before scanning.

Suggested change

Where("user_challenges.user_id IN ? AND user_challenges.solved = ?", topUserId, true).

Where("user_challenges.user_id IN ? AND user_challenges.solved = ?", topUserId, true).

Order("user_challenges.user_id, user_challenges.challenge_id, user_challenges.created_at ASC").

Copilot · 2026-04-13T09:52:14Z

+[instance_config]
+# Port Range for localhost. per-server this is configured via `port-range`
+local_host_port_range = '10000:11000'
+default_expiration = 300
+max_extension = 600
+max_instances_per_user = 3
+


The [instance_config] section documents local_host_port_range, but the code only supports per-server available_servers.<name>.port_range and does not read local_host_port_range. This mismatch means copying the example config can leave localhost/servers without a usable port range, breaking instance/compose deployment. Align the example and code: either remove this key from docs or add support + fallback logic in config validation.

Copilot · 2026-04-13T09:52:15Z

 // ParsePortMapping parses the port mapping string and return the required ports
 // If the portMapping string is not valid, this returns an error.
-// The format of the port mapping is `HOST_PORT:CONTAINER_PORT`
+// The format of the port mapping is `PORT_FIRST:PORT_LAST`
 func ParsePortMapping(portMap string) (uint32, uint32, error) {
-	ports := strings.Split(portMap, mappingDelimeter)
+	ports := strings.Split(portMap, core.MappingDelimeter)

 	if len(ports) != 2 {
 		return 0, 0, errors.New("port mapping string is not valid")
 	}

-	hostPort, err := strconv.ParseUint(ports[0], 10, 32)
+	firstPort, err := strconv.ParseUint(ports[0], 10, 32)
 	if err != nil {
 		return 0, 0, fmt.Errorf("host port is not a valid port in: %s", portMap)
 	}

-	containerPort, err := strconv.ParseUint(ports[1], 10, 32)
+	lastPort, err := strconv.ParseUint(ports[1], 10, 32)
 	if err != nil {
 		return 0, 0, fmt.Errorf("container port is not a valid port in: %s", portMap)
 	}

-	return uint32(hostPort), uint32(containerPort), nil
+	return uint32(firstPort), uint32(lastPort), nil


ParsePortMapping now parses a port range (PORT_FIRST:PORT_LAST), but the error messages still refer to “host port” / “container port”, which is confusing for callers and operators. Update the messages to reference “first port” / “last port” (and ideally validate firstPort <= lastPort here too, so all callers get consistent validation).

gqvz mentioned this pull request Feb 5, 2026

Add Redis cache #445

Closed

gqvz force-pushed the redis-instance branch from d6eb84b to 5656e50 Compare February 6, 2026 11:49

gqvz requested a review from Copilot February 6, 2026 11:51

Copilot started reviewing on behalf of gqvz February 6, 2026 11:51 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

kunrex force-pushed the redis-instance branch 3 times, most recently from 9786785 to 42be447 Compare February 9, 2026 16:20

v1bh475u requested changes Mar 5, 2026

View reviewed changes

kunrex force-pushed the redis-instance branch from e9f9f75 to 7904677 Compare April 2, 2026 14:30

gqvz and others added 19 commits April 13, 2026 13:02

Add Empty Spawn Instance Handler

8d18c80

Add redis cache backup and restore commands

4ddd5cb

Refactor Redis command execution to use environment variable for pass…

6075e1e

…word

Enhance Redis backup command to conditionally include user and passwo…

5b43dce

…rd arguments

Cleanup: add cache cleanup

0714508

Feat: Add redis set up to beast init and beast config

3022211

Rename Cache function

db539d0

Add graceful cache closure

57f8aad

Add redis set up to config

047b97e

Add acl save and fix sprintf typos

4cb8eea

Add cache comment and acl warning

5312624

Add instanced challenges

78a9107

Init key utils

00a5cb1

Erros: Add better error messages

d2e3a99

Cleanup: Fix typos, clean comments, add constants for magic values.

ea1ed3a

Remove hosted address

ec8b944

Refactor instance manager

eb7a6f0

Abstract functions to pkg

9555cca

gqvz and others added 4 commits April 13, 2026 13:03

Add instanced info to challenge metadata

3dcf7c0

Fix compose deploy project name

1b4b7c0

Add dynamic port mapping to docker compose challenges

d3e1c0a

Resolve server host issue

89b96b0

kunrex force-pushed the redis-instance branch from dd2f9b5 to 89b96b0 Compare April 13, 2026 07:53

gqvz and others added 4 commits April 13, 2026 14:10

refactor: implement distinct user challenge tracking with accurate so…

28e2a11

…lve timestamps and optimized database queries

refactor: optimize slice allocation

83481d0

feat: integrate hint penalties into leaderboard calculations

2d03aa1

Fix port assign bug

5f3effb

v1bh475u requested a review from Copilot April 13, 2026 09:45

Copilot started reviewing on behalf of v1bh475u April 13, 2026 09:45 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

kunrex added 10 commits April 13, 2026 15:45

Fix port bug suggestions

8f5ee9a

Rework port management to avoid unnececarry race conditions

70b679b

Add server deployed to api

4ffd6e8

Add order by to query top users

584e4eb

Add error check to save flag submission

5ed55dd

Avoid default port mapping in compose

d54ff4f

Fix kill instance container compose project name

4be8f6a

Add User Hint migration

b28730c

Refactor tag fetching from DB

0f26f37

Update exaples

2842db8

kunrex force-pushed the redis-instance branch from c4eb063 to 2842db8 Compare April 14, 2026 10:41

kunrex added 6 commits April 14, 2026 16:23

Fix instance port assignment

4cde860

Add resource limits to containers

9b2a826

Fix instance container ports

4f266f8

Unify localhost checks

f8a43db

Fix container name bugs

32433a3

Kill instances for running challenges

6d12008

kunrex force-pushed the redis-instance branch from 9624fa0 to 6d12008 Compare April 16, 2026 07:07

		# DO NOT specify ports for instanced challenges!
		# Instead, use default_port to indicate which container port to expose

		// Postgresql database for beast. The Db variable is the connection variable for the
		// database, which is not closed after creating a connection here and can

	Where("user_challenges.user_id IN ? AND user_challenges.solved = ?", topUserId, true).
	Where("user_challenges.user_id IN ? AND user_challenges.solved = ?", topUserId, true).
	Order("user_challenges.user_id, user_challenges.challenge_id, user_challenges.created_at ASC").

Conversation

gqvz commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gqvz commented Feb 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

v1bh475u Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

v1bh475u Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

v1bh475u Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

kunrex Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gqvz commented Feb 5, 2026 •

edited

Loading