JavaScript heap out of memory for 10s of MB of source

[taoqf]

TypeScript Version: 2.4.1 / nightly (2.5.0-dev.20170707)

Code
gulpfile.js

// A *self-contained* demonstration of the problem follows...
gulp.task('compile-ts', () => {
	const ts = require('gulp-typescript');
	const tsProject = ts.createProject('./tsconfig.json');
	tsProject.options.module = 1;
	const dest = tsProject.options.outDir;
	return gulp.src('./src/**/*.ts')
		.pipe(tsProject())
		.pipe(gulp.dest(dest));
});

tsconfig.json

{
        "include": [
                "src"
        ],
        "compilerOptions": {
                "module": "commonjs",
                "target": "esnext",
                "noImplicitAny": false,
                "sourceMap": false,
                "emitDecoratorMetadata": true,
                "experimentalDecorators": true,
                "strictNullChecks": false,
                "noImplicitThis": true,
                "rootDir": "./src/",
                "rootDirs": [],
                "allowJs": false,
                "allowUnreachableCode": false,
                "allowUnusedLabels": false,
                "alwaysStrict": true,
                "baseUrl": "",
                "charset": "utf8",
                "declaration": false,
                "inlineSourceMap": false,
                "allowSyntheticDefaultImports": false,
                "diagnostics": false,
                "emitBOM": false,
                "forceConsistentCasingInFileNames": false,
                "importHelpers": false,
                "inlineSources": false,
                "isolatedModules": false,
                "lib": [
                        "es6",
                        "es7",
                        "dom"
                ],
                "listFiles": true, // default false
                "listEmittedFiles": true, // default false
                "locale": "zh_CN",
                "newLine": "CRLF",
                "noEmit": false,
                "moduleResolution": "node",
                "noEmitHelpers": false,
                "noEmitOnError": false,
                "noImplicitReturns": false,
                "noImplicitUseStrict": false,
                "maxNodeModuleJsDepth": 0,
                "noLib": false,
                "outDir": "./dist/",
                "noFallthroughCasesInSwitch": false,
                "noResolve": false,
                "noUnusedLocals": false,
                "noUnusedParameters": false,
                "paths": {},
                "preserveConstEnums": false,
                "pretty": true,
                "removeComments": false,
                "skipDefaultLibCheck": true, // default false
                "skipLibCheck": true, // default false
                "stripInternal": false,
                "suppressExcessPropertyErrors": false,
                "suppressImplicitAnyIndexErrors": true, // default false
                "traceResolution": true, // default false
                "typeRoots": [],
                "types": [],
                "watch": false
        }
}

Expected behavior:

Actual behavior:
Exception i got:

<--- Last few GCs --->

[24442:0x2def9d0]    58440 ms: Mark-sweep 1400.1 (1465.9) -> 1400.1 (1465.9) MB, 1948.3 / 0.0 ms  allocation failure GC in old space requested
[24442:0x2def9d0]    60312 ms: Mark-sweep 1400.1 (1465.9) -> 1400.1 (1434.9) MB, 1871.6 / 0.0 ms  last resort 
[24442:0x2def9d0]    62184 ms: Mark-sweep 1400.1 (1434.9) -> 1400.0 (1434.9) MB, 1871.8 / 0.0 ms  last resort 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3d1f48829891 <JS Object>
    2: write [/home/taoqf/feidao/temp/boai-h5-6/1/node_modules/typescript/lib/typescript.js:~9035] [pc=0x3cfdedb6fe70](this=0x1857e443e689 <an Object with map 0xddd4399bc9>,s=0x3c6749682c29 <String[4]: "id">)
    3: pipelineEmitExpression [/home/taoqf/feidao/temp/boai-h5-6/1/node_modules/typescript/lib/typescript.js:~66807] [pc=0x3cfdedbd085f](this=0xf51a038dbf9 <JS Global Object>,node=0x29110b...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [gulp]
 2: 0x13647ec [gulp]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [gulp]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [gulp]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [gulp]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [gulp]
 7: 0x3cfded28437d
Aborted

More

• tsc works fine.
• do not depend on fp-ts, fantasyland and monet
• if i compile my ts files separately(1 ts file per time), also there is no exception.
• i have searched all the issues already and found no clue. like FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory #14628 (comment)

[taoqf]

@sandersn , Could this be fixed?

[sandersn]

Two questions:

1. Can you provide the code or a small repro?
2. The gulp-typescript setup changes options.module=1. Does tsc still succeed when you set the equivalent module option in tsconfig? It's commonjs right now and I'm not sure that 1 means commonjs

Afterwards, if tsc still works, then it sounds like a gulp-typescript bug. The stack is in emit when it runs out of memory. This is very unusual for vanilla tsc, so I would first suspect a configuration/setup bug in gulp-typescript.

Also, @rbuckton, does that crash location mean anything to you?

[taoqf]

1 means comonjs 2 amd 3 umd
I set options.module=1 this because i have multi tasks in gulp.
I guess gulp may use more memory than tsc because it will store files in memory steam.
I have tested if I reduce the number of the file it will succeed.

[mhegazy]

looks like the gulp typescript + typescript memory usage pushes you over the limit. I would say gulp-typescript should take the first stab at this issue.

[taoqf]

@mhegazy , I got this Error just now:

<--- Last few GCs --->

[22042:0x3696020]    34073 ms: Mark-sweep 1403.3 (1447.3) -> 1403.3 (1442.3) MB, 877.7 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 878 ms) last resort 
[22042:0x3696020]    34966 ms: Mark-sweep 1403.3 (1442.3) -> 1403.3 (1442.3) MB, 891.5 / 0.0 ms  last resort 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2a16e0b9cef1 <JSObject>
    1: declareSymbol [/home/taoqf/.nvm/versions/node/v8.4.0/lib/node_modules/typescript/lib/tsc.js:~18571] [pc=0x1b65f5d1bf8b](this=0x3f86308890d9 <JSGlobal Object>,symbolTable=0x10f596065381 <Map map = 0x17f790d183b9>,parent=0x220cdb20d551 <Symbol map = 0x17f790d19019>,node=0x14238561b021 <Node map = 0x17f790d14e19>,includes=4,excludes=0)
    2: bindPropertyOrMethodOrAccessor [/home/taoqf/.nvm/...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x13740dc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 7: 0x1b65f58040dd
[1]    22042 abort      tsc

using tsc.

[mhegazy]

can you share your project

[swlee60]

+1

I have same problem.
I can't share my project because our enterprise project.
This is not appear in normal compile environment, but almost appear in below environment.

1. large or huge size project
2. If you change the file frequently during typescript compile (with hot module replacement), a crash appears.