feat: Add pool support in process package#1215

krishankumar01 · 2025-10-03T08:06:01Z

📑 Description

This PR implements a lightweight, flexible API for managing concurrent processes in Go.

Basic Usage

// Run multiple commands
results, err := process.NewPool().Run(func(p process.Pool) {
    p.Command("echo", "hello").As("greeting")
    p.Command("ls", "-la").As("files")
})

fmt.Println(results["greeting"].Output())

Features

Run multiple commands concurrently
Control concurrency limits
Stream output in real-time
Set timeouts and contexts
Send signals to running processes

Examples

Limit Concurrency

// Run max 2 commands at once
process.NewPool().Concurrency(2).Run(func(p process.Pool) {
    p.Command("task1").As("job1")
    p.Command("task2").As("job2")
    p.Command("task3").As("job3")
})

Stream Output

process.NewPool().OnOutput(func(key string, typ process.OutputType, line []byte) {
    fmt.Printf("[%s] %s", key, string(line))
}).Run(func(p process.Pool) {
    p.Command("slow-command").As("slow")
})

Control Running Processes

running, err := process.NewPool().Start(func(p process.Pool) {
    p.Command("server").As("server")
})

pids := running.PIDs()

running.Signal(syscall.SIGTERM)

results := running.Wait()

Command Options

p.Command("cmd").
    Path("/custom/dir").                     // Working directory
    Env(map[string]string{"KEY": "value"}).  // Environment variables
    Timeout(5 * time.Second).                // Command timeout
    WithContext(ctx).                        // Command context
    DisableBuffering().                      // Don't buffer output
    Quietly().                               // Suppress output
    As("name")                               // Set command name

Notes

Use DisableBuffering() for commands with large output
Command-level contexts override pool-level contexts

✅ Checks

Added test cases for my code

Copilot

Pull Request Overview

This PR introduces a process pool functionality to the Goravel framework and removes deprecated LatestOutput and LatestErrorOutput methods. The changes add concurrent process execution capabilities with configurable scheduling, timeout management, and output handling.

Adds a comprehensive process pool implementation with builder pattern for concurrent process execution
Removes deprecated LatestOutput and LatestErrorOutput methods from the Running interface and implementation
Introduces scheduling strategies and priority-based execution for pool commands

Reviewed Changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
process/running_unix_test.go	Removes test for deprecated LatestOutput functionality
process/running.go	Removes deprecated LatestOutput and LatestErrorOutput methods and helper function
mocks/process/Running.go	Removes mock methods for deprecated LatestOutput functionality
contracts/process/running.go	Removes deprecated method signatures from Running interface
process/pool.go	Adds complete process pool implementation with builder pattern and worker management
process/running_pool.go	Adds RunningPool implementation for managing active process pools
contracts/process/pool.go	Defines interfaces for pool builder, pool, and pool commands
contracts/process/running_pool.go	Defines RunningPool interface for pool lifecycle management
contracts/process/schedule.go	Defines scheduling interfaces and priority system for process execution

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

process/pool.go

contracts/process/schedule.go

contracts/process/pool.go

codecov · 2025-10-14T16:00:03Z

Codecov Report

❌ Patch coverage is 94.82072% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.35%. Comparing base (a136858) to head (a2a79d6).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
process/running_pool.go	84.78%	4 Missing and 3 partials ⚠️
process/pool.go	96.29%	4 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1215      +/-   ##
==========================================
+ Coverage   65.95%   66.35%   +0.40%     
==========================================
  Files         243      244       +1     
  Lines       16698    16895     +197     
==========================================
+ Hits        11013    11211     +198     
+ Misses       5304     5302       -2     
- Partials      381      382       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

#567-3

Copilot

Pull Request Overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

hwbrzzl

Great, only a few nitpicks.

process/pool.go

hwbrzzl · 2025-10-17T02:24:05Z

process/pool.go

+	return run.Wait(), nil
+}
+
+func (r *PoolBuilder) start(configure func(contractsprocess.Pool)) (contractsprocess.RunningPool, error) {


Could you add some annotations for this function logic?

hwbrzzl · 2025-10-17T11:11:02Z

contracts/process/pool.go

+	Run(builder func(Pool)) (map[string]Result, error)
+
+	// Start launches the pool asynchronously and returns a handle to the running
+	// pool, allowing for interaction with the live processes.
+	Start(builder func(Pool)) (RunningPool, error)


Can they be implemented like below?

That would be a bit complicated and not really necessary. We could add a Pool helper and allow passing OnOutput directly in the Start method, but since most of our existing implementations already use an OnOutput callback, it’s better to stay consistent with that approach.

Below is a tidy sample that implements the PHP example in our style:

package main func main() { running, err := process.NewPool(). OnOutput(func(key string, typ process.OutputType, line []byte) { // ... }). Start(func(p process.Pool) { p.Command("bash", "import-1.sh").Path("./") p.Command("bash", "import-2.sh").Path("./") p.Command("bash", "import-3.sh").Path("./") }) if err != nil { os.Exit(1) } <-running.Done() results := running.Wait() _ = results }

OnOutput is fine, can it be?

package main func main() { running, err := facades.Process.Pool(func(p process.Pool) { p.Command("bash", "import-1.sh").Path("./") p.Command("bash", "import-2.sh").Path("./") p.Command("bash", "import-3.sh").Path("./") }). OnOutput(func(typ process.OutputType, line []byte, key string) { // ... }). Start() if err != nil { os.Exit(1) } <-running.Done() results := running.Wait() _ = results }

nit: I updated the sort of OnOutput func(typ process.OutputType, line []byte, key string) to be consistent with Laravel.

Yeah, I’ll think about refactoring it from a facade perspective. Still not sure how it’ll be used with facades though. I’ll raise a new PR for that and include the similar contract changes there itself.

Thanks, it can be optimized in another PR if you want.

process/running_pipe.go

hwbrzzl

Amazing 👍

process/pipeline.go

hwbrzzl · 2025-10-17T14:30:07Z

contracts/process/pool.go

+	Run(builder func(Pool)) (map[string]Result, error)
+
+	// Start launches the pool asynchronously and returns a handle to the running
+	// pool, allowing for interaction with the live processes.
+	Start(builder func(Pool)) (RunningPool, error)


Thanks, it can be optimized in another PR if you want.

hwbrzzl

Thanks

krishankumar01 added 2 commits October 1, 2025 16:29

remove LatestOutput and LatestErrorOutput methods

6dd217c

implement pool for processes

f90a4b3

krishankumar01 requested a review from a team as a code owner October 3, 2025 08:06

Copilot AI review requested due to automatic review settings October 3, 2025 08:06

Copilot AI reviewed Oct 3, 2025

View reviewed changes

process/pool.go Outdated Show resolved Hide resolved

krishankumar01 changed the title ~~Kkumar gcc/#567 3~~ feat: Add pool support in process package Oct 3, 2025

krishankumar01 marked this pull request as draft October 3, 2025 08:16

hwbrzzl reviewed Oct 3, 2025

View reviewed changes

contracts/process/schedule.go Outdated Show resolved Hide resolved

hwbrzzl reviewed Oct 3, 2025

View reviewed changes

contracts/process/pool.go Show resolved Hide resolved

hwbrzzl reviewed Oct 3, 2025

View reviewed changes

contracts/process/pool.go Show resolved Hide resolved

krishankumar01 and others added 2 commits October 14, 2025 21:26

remove strategy

6a20d09

Merge branch 'master' into kkumar-gcc/#567-3

cf3828e

krishankumar01 and others added 5 commits October 16, 2025 23:55

add test cases for pool

9d993f0

Merge branch 'master' into kkumar-gcc/#567-3

bd5d7b6

fix linting error

a6badad

Merge remote-tracking branch 'origin/kkumar-gcc/#567-3' into kkumar-gcc/

2608dfb

#567-3

fix linting error

614e591

krishankumar01 marked this pull request as ready for review October 16, 2025 18:49

krishankumar01 requested review from Copilot and hwbrzzl October 16, 2025 18:50

Copilot AI reviewed Oct 16, 2025

View reviewed changes

hwbrzzl reviewed Oct 17, 2025

View reviewed changes

krishankumar01 added 2 commits October 17, 2025 17:48

rename steps to commands

16313d9

add implementation comments for start

641fe50

hwbrzzl previously approved these changes Oct 17, 2025

View reviewed changes

rename steps to commands

51f7ead

krishankumar01 dismissed hwbrzzl’s stale review via 51f7ead October 17, 2025 14:35

krishankumar01 added 2 commits October 17, 2025 23:26

change signature of OnOutputFunc variations

d839e68

change signature of OnOutputFunc variations

a2a79d6

hwbrzzl approved these changes Oct 18, 2025

View reviewed changes

krishankumar01 merged commit 9ebd55c into master Oct 18, 2025
14 checks passed

krishankumar01 deleted the kkumar-gcc/#567-3 branch October 18, 2025 02:39

krishankumar01 mentioned this pull request Oct 26, 2025

feat: add process facade #1232

Merged

1 task

Conversation

krishankumar01 commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📑 Description

Basic Usage

Features

Examples

Limit Concurrency

Stream Output

Control Running Processes

Command Options

Notes

✅ Checks

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

hwbrzzl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hwbrzzl Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hwbrzzl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hwbrzzl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

krishankumar01 commented Oct 3, 2025 •

edited

Loading

codecov bot commented Oct 14, 2025 •

edited

Loading

hwbrzzl Oct 17, 2025 •

edited

Loading