Matlab Setup Worker State

Set Up Worker State for Parallel Pools

By default, process workers in a parallel pool inherit MATLAB path state from

their controlling client, but they do not inherit loaded libraries, open

connections, or expensive pre-computed objects. Code that relies on any of

these needs explicit setup. This skill teaches the correct APIs for each

scenario. Most patterns target process-based pools; thread pool applicability

is noted where relevant.

When to Use

• Code needs a non-serializable resource on workers (database connections,

COM objects, loaded shared libraries, file handles)

• Code needs expensive one-time setup per worker (large object construction,

data loading) that should not repeat every parfor iteration

• Code needs paths or environment variables set on workers
• User has existing code using spmd for side-effect setup before parfor

(anti-pattern — help them modernise)

• User sees errors about objects not being serializable when passed to parfor

When NOT to Use

• Data already in client memory used in a single parfor loop (MATLAB broadcasts it automatically — Constant adds complexity for no benefit in this case)
• Choosing between process and thread pools (out of scope — this skill assumes a pool type is already chosen)

Decision Framework

| Scenario | Correct API | Why |

|----------|-------------|-----|

| Non-serializable resource needing cleanup (connections, libraries) | parallel.pool.Constant(@buildFcn, @cleanupFcn) | Constructs on each worker; automatic cleanup on delete |

| Data from a file needed on workers | parallel.pool.Constant(@() load(file).var) | Default for file-based data. Each worker loads from disk; client never holds the dataset. Always prefer this when the source is a file. |

| Data already in client memory, used in multiple parfor loops | parallel.pool.Constant(data) | Transfers once for pool lifetime; without Constant, broadcast re-sends for every parfor loop. Not needed for a single parfor — let MATLAB broadcast. |

| One-shot side effect, no return value needed | parfevalOnAll(pool, @fcn, 0) | Runs once on all workers; use fetchOutputs to surface errors |

| Paths needed on workers — parpool call is in your code | parpool(..., AdditionalPaths=paths) | Cleanest option when you can modify the parpool call; client path entries are inherited automatically |

| Paths needed on workers — pool opened elsewhere (can't modify call) | parfevalOnAll(pool, @addpath, 0, p) | Never delete and recreate a pool just to add paths — use parfevalOnAll on the existing pool |

| Environment variables on workers — parpool call is in your code | parpool(..., EnvironmentVariables=vars) | Forwards named env vars from client to workers at startup; set values with setenv on the client before this call |

| Environment variables on workers — pool opened elsewhere (can't modify call) | parfevalOnAll(pool, @setenv, 0, k, v) | Never delete and recreate a pool just to set env vars — use parfevalOnAll on the existing pool |

Thread pool notes: parallel.pool.Constant and parfevalOnAll also work on

thread pools. However, thread workers share the client's process, so path and

environment variable changes on the client are visible to threads automatically —

AdditionalPaths and EnvironmentVariables do not apply. To modify a thread

worker's environment, alter the client environment before the parfor.

The spmd Anti-Pattern

This applies equally to process pools and thread pools — prefer

parallel.pool.Constant over spmd for worker state setup regardless of pool

type.

What it looks like

% ANTI-PATTERN: Do not do this
spmd
    loadlibrary("mylib", "mylib.h");
end
parfor i = 1:100
    result(i) = calllib("mylib", "compute", data(i));
end
spmd
    unloadlibrary("mylib");
end

Why it's fragile

• No cleanup guarantee — if the parfor errors, the second spmd never runs
• Makes the pool fragile to worker disconnection — once a pool has run an

spmd block, a single worker losing connection tears down the entire pool.

Pools that never use spmd continue operating with fewer workers if one

disconnects (no replacement occurs — the pool simply shrinks).

• Composite variables cannot be used inside parfor — values created in

spmd are Composite objects that require indexing on the client; they

cannot be referenced directly from within a parfor body

How to modernise

Replace with parallel.pool.Constant:

C = parallel.pool.Constant( ...
    @() loadAndReturn(), ...
    @(~) unloadlibrary("mylib")); %#ok<NASGU> — kept alive for cleanup

result = zeros(1, 100);
parfor i = 1:100
    result(i) = calllib("mylib", "compute", data(i));
end

% Cleanup happens automatically when C goes out of scope

Or with parfevalOnAll for simple side effects:

f = parfevalOnAll(pool, @setupWorker, 0);
fetchOutputs(f);

result = zeros(1, 100);
parfor i = 1:100
    result(i) = doWork(data(i));
end
% No automatic cleanup

Prefer parallel.pool.Constant when cleanup is needed. Use parfevalOnAll

only for operations where cleanup is not required.

Patterns

Pattern 1: Loading data from a file for use in parfor

Always use the build-function form when the source is a file. This avoids

loading the data into client memory entirely — each worker loads directly from

disk. This is critical for large files but is also the correct default for any

file-based data because it scales without code changes as file size grows.

c = parallel.pool.Constant(@() load("costSurface.mat").costSurface);

results = zeros(1, 10000);
parfor i = 1:10000
    results(i) = processWithLookup(input(i), c.Value);
end

Each worker calls load() once. The client never holds the full dataset.

Do NOT do this — loading on the client then wrapping defeats the purpose:

% WRONG: loads entire file into client memory, then copies to each worker
data = load("costSurface.mat").costSurface;
c = parallel.pool.Constant(data);  % client already paid the memory cost

The only time parallel.pool.Constant(data) (without a build function) is

appropriate is when the data is already in client memory and used across

multiple parfor loops (avoids re-broadcasting each loop). For a single parfor

loop, just let MATLAB broadcast — Constant adds no benefit.

Pattern 2: Shared library (non-serializable, needs cleanup)

c = parallel.pool.Constant( ...
    @() loadLibraryAndReturn("mylib", "mylib.h"), ...
    @unloadlibrary);

results = zeros(1, 100);
parfor i = 1:100
    results(i) = calllib(c.Value, "compute", data(i));
end
% Cleanup runs automatically when c goes out of scope

function libName = loadLibraryAndReturn(libName, headerFile)
    loadlibrary(libName, headerFile);
end

The build function returns the library name so that c.Value is naturally

referenced in the parfor body — this triggers lazy initialisation. The cleanup

function receives the same value and calls unloadlibrary on it.

Pattern 3: One-shot setup with parfevalOnAll (no cleanup needed)

pool = gcp;
f = parfevalOnAll(pool, @setupWorker, 0);
fetchOutputs(f);

Pattern 4: Database connection (use createConnectionForPool)

Database connections are a common specific case that requires Database

Toolbox. Prefer the purpose-built helper

createConnectionForPool

over a hand-rolled parallel.pool.Constant. It returns a

parallel.pool.Constant whose value is a per-worker connection to the

configured data source, and it handles worker-side initialisation that a naive

database() call inside a Constant build function does not.

pool = gcp;
c = createConnectionForPool(pool, "MyDataSource", "", "");

results = cell(1, 1000);
parfor i = 1:1000
    conn = c.Value;
    results{i} = fetch(conn, sprintf("SELECT val FROM t WHERE id=%d", i));
end

The returned Constant is the standard parallel.pool.Constant type, so it

participates in the normal lifetime model: connections are torn down when the

Constant goes out of scope or the pool shuts down.

Recognising When pool.Constant Is Needed

Look for these signals in user code:

| Signal | Indicates |

|--------|-----------|

| Object created and destroyed every parfor iteration | Per-worker resource needed |

| "Cannot serialize" or "not valid on workers" errors | Non-serializable resource |

| loadlibrary/unloadlibrary inside parfor | Library should persist across iterations |

| database()/close() inside parfor | Connection should persist |

| fopen/fclose inside parfor (same file) | File handle should persist |

| Large variable used read-only in multiple parfor loops (>100 MB) | Send-once with Constant |

| spmd block doing setup before parfor | Modernise to Constant or parfevalOnAll |

Common Mistakes

| What the agent does wrong | Why it's wrong | Correct approach |

|---------------------------|---------------|------------------|

| Uses spmd before parfor for setup | Fragile, no cleanup, pool tears down if any worker disconnects | parallel.pool.Constant with build + cleanup functions |

| Creates resource every iteration | Massive overhead (N connections for N iterations) | parallel.pool.Constant — one per worker |

| Uses persistent variables on workers | No cleanup, hard to reason about lifecycle | parallel.pool.Constant — explicit lifecycle |

| Tries to send non-serializable object from client | Will error — can't serialize connections, libraries | Use build function form: parallel.pool.Constant(@buildFcn) |

| Build function calls a function not on the worker path | Workers can't find it — "Unable to create parallel.pool.Constant on the workers" | Ensure build function helpers are on the path (or use AdditionalPaths). Local functions in the same file work — MATLAB captures them with the handle. |

| Uses addpath inside parfor on process pool | Runs every iteration unnecessarily | parpool(..., AdditionalPaths=...) or parfevalOnAll |

| Deletes and recreates pool to change paths or env vars | Extremely expensive — pool startup can take 30+ seconds and may queue on a scheduler | Use parfevalOnAll on the existing pool instead |

Key Functions

| Function | Product | Available From | Purpose |

|----------|---------|----------------|---------|

| parallel.pool.Constant | Parallel Computing Toolbox | R2015b | Per-worker state with lifecycle management |

| parfevalOnAll | Parallel Computing Toolbox | R2013b | Run a function once on all workers |

| parpool(..., AdditionalPaths=) | Parallel Computing Toolbox | R2025a | Set worker paths at pool creation |

| parpool(..., EnvironmentVariables=) | Parallel Computing Toolbox | R2017b | Forward env vars to workers at pool creation |

| createConnectionForPool | Database Toolbox | R2019a | Per-worker database connections (optional) |

Conventions

• Prefer parallel.pool.Constant over spmd for any worker state that persists across parfor iterations
• The build-function form (parallel.pool.Constant(@buildFcn)) is lazily initialised — the build function does not run until c.Value is first accessed on a worker. Always reference c.Value inside the parfor body to ensure initialisation occurs.
• Always provide a cleanup function when the resource needs explicit teardown
• Use parfevalOnAll only when no handle is needed back and no cleanup is required
• Use parpool name-value pairs when the setup is needed for the entire pool lifetime (paths, env vars)
• When modernising spmd-based setup code, preserve the original's intent (what resource, what cleanup) while switching to Constant

----

Copyright 2026 The MathWorks, Inc.

----