[edalzell]

Index::update() deletes the index and dispatches one InsertMultipleJob per chunk_size documents. On the local driver each job's insertDocuments() reads, decodes, re-encodes and rewrites the whole index file, so a reindex of 10,000 entries at the default chunk size rewrites a file growing to 25 MB a hundred times. On a sync queue (the default install, and php please search:update on most sites) all of that happens inline in one process.

Measured on a sync queue, median of 3 (scripts: https://gist.github.com/edalzell/06252c75c5c8d9bd14ab9f94a2041350). update() is the full reindex including loading entries; "chunk replay" is only the file work the chunk jobs do; "single write" is the same documents written once.

docs	file	update() before	update() after	chunk replay	single write
2,001	5 MB	333 ms	213 ms	142 ms	4 ms
10,000	25 MB	4,650 ms	1,183 ms	3,500 ms	22 ms

Fix: Comb\Index::update() collects what the inline chunk jobs insert and writes the file once after parent::update(). IndexManager caches index instances, so the sync job resolves the same instance that is mid-update. On a real queue the jobs run in other processes, the buffer stays empty and nothing changes; under Bus::fake() the index is still deleted and nothing is written, so the existing dispatch tests pass untouched. Tests: 250 entries on a sync queue produce one put with all 250 documents; a faked bus writes nothing; inserts outside update() still write immediately.

[duncanmcclean]

Can you please resolve the merge conflicts?

[edalzell]

Can you please resolve the merge conflicts?

Done