Improve memory usage for large files

[alexdima]

e.g.

• open the file and wait for ~ 30s

---

1. file A

Download Loan Data for 2016 Q3 from https://www.lendingclub.com/info/download-data.action

	empty file	73 MB 99127 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	353 MB	163 MB	90 MB	952.0 B
OSX Heap Snapshot	50 MB	138 MB	88 MB	15 MB	158.6 B

---

2. file B

Download checker.ts from https://github.com/Microsoft/TypeScript/blob/296660a2a077c32a2ed41cb762ef530031e56417/src/compiler/checker.ts

Generate a 128x times checker:

var fs = require('fs');
var str = fs.readFileSync('checker.ts').toString();
for (var i = 0; i < 7; i++) {
    str = str + '\n' + str;
}
fs.writeFileSync('checker-out.ts', str);

	empty file	177 MB 3171200 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	1474 MB	1284 MB	1107 MB	366.0 B
OSX Heap Snapshot	50 MB	716 MB	666 MB	489 MB	161.7 B

---

• OSX shallow size taken by a ModelLine (via Heap Snapshot) - 72 bytes

[alexdima]

Removing ModelLine._lineNumber

• OSX shallow size taken by a ModelLine (via Heap Snapshot) - 64 bytes

1. file A

	empty file	73 MB 99127 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	348 MB	158 MB	85 MB	899.1 B
OSX Heap Snapshot	50 MB	136 MB	86 MB	13 MB	137.51 B

2. file B

	empty file	177 MB 3171200 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	1194 MB	1004 MB	827 MB	273.45 B
OSX Heap Snapshot	50 MB	692 MB	642 MB	465 MB	153.75 B

[alexdima]

Using MinimalModelLine

• OSX shallow size taken by a MinimalModelLine (via Heap Snapshot) - 40 bytes

1. file A

	empty file	73 MB 99127 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	347 MB	157 MB	84 MB	888.6 B
OSX Heap Snapshot	50 MB	133 MB	83 MB	10 MB	105.8 B

2. file B

	empty file	177 MB 3171200 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	1078 MB	888 MB	711 MB	235.1 B
OSX Heap Snapshot	50 MB	577 MB	527 MB	350 MB	115.7 B

[alexdima]

Using MinimalModelLine with lazy markers

• OSX shallow size taken by a MinimalModelLine (via Heap Snapshot) - 40 bytes or 32 bytes

1. file A

	empty file	73 MB 99127 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	348 MB	158 MB	85 MB	899.1 B
OSX Heap Snapshot	50 MB	132 MB	82 MB	9 MB	95.2 B

2. file B

	empty file	177 MB 3171200 lines	actual usage	overhead	overhead per line
OSX RESIDENT	190 MB	991 MB	801 MB	624 MB	206.3 B
OSX Heap Snapshot	50 MB	553 MB	503 MB	326 MB	107.8 B