Skip to content

replace/replaceAll never terminate on a global regexp that can match the empty string #336

Description

@theRizwan

Version

v1.2.2 / master @ 1bb4cce

What happens

_replaceRegexp collects matches by driving exec in a loop:

while (true) {
  const match = re.exec(str)
  if (!match) break
  matches.push(match)
}

A zero-length match does not advance lastIndex, so the loop rematches at the
same index forever. Any global regexp that can match the empty string never
terminates — it allocates matches until the process runs out of memory:

import MagicString from 'magic-string'

new MagicString('a\nb\nc').replaceAll(/^/gm, '// ')
// FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

/^/gm, /$/gm, /\b/g, /\s*/g, /x?/g and /a*/g are all affected, so
prefixing or suffixing every line cannot be done through replaceAll today.
String.prototype handles all of them:

'a\nb\nc'.replaceAll(/^/gm, '// ') // '// a\n// b\n// c'

Second, smaller problem in the same loop

lastIndex is never reset before the scan, so a global regexp that has already
been used resumes from wherever it stopped and silently skips earlier matches.
String.prototype.replace always starts from the beginning of the string:

const re = /o/g
re.exec('foo')                             // lastIndex is now 2

'foo'.replaceAll(re, 'X')                  // 'fXX'
new MagicString('foo').replaceAll(re, 'X') // 'foX'  <- 'fo' was skipped

Nothing is thrown here, the wrong characters are simply left alone.

Non-global and string overloads

Once the loop terminates, an empty match reaches overwrite, which rejects a
zero-length range, so the non-global path throws rather than hangs:

new MagicString('abc').replace(/x?/, 'Y')
// [MagicString] cannot overwrite a zero-length range at 0, use appendLeft() or prependRight()
'abc'.replace(/x?/, 'Y') // 'Yabc'

The string overloads have the same gap. replace('', x) throws as above, and
replaceAll('', x) cannot terminate either, for a different reason — indexOf
clamps its start index to the string length, so original.indexOf('', 4) on a
3-character string returns 3 rather than -1 and the loop never ends:

'abc'.replace('', 'X')    // 'Xabc'
'abc'.replaceAll('', 'X') // 'XaXbXcX'

Expected

An empty match spans no characters, so there is no range to overwrite — the
substitution is an insertion at the matched position, which is what
String.prototype.replace does with one. The README lists only two differences
from String.replace ("always match against the original string", "mutates the
magic string state") and empty matches are not among them.

So: terminate, start global regexps from the beginning of the string, and insert
at a zero-length match.

I have a PR ready for this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions