Example parsers

[Geal]

We currently have a few example parsers. In order to test the project and make it useful, other formats can be implemented. Here is a list, if anyone wants to try it:

• text file formats:
  • INI
  • FASTQ
  • libconfig-like configuration file format
  • torrc configuration file
  • ISO 8601 dates
  • Web archive
  • TOML
  • bencode
  • CSV
  • YAML
  • CommonMark
• audio, video and image formats:
  • MP4 (partial implementation)
  • GIF
  • FLAC
  • FLV
  • MKV
  • OGG
  • MPEG TS
  • AVI
  • PNG
  • JPEG
  • EXIF
  • MP3
• document formats:
  • torrent files
  • TAR
  • PDF
  • MS-CFB (compound format, used in doc, xls, ppt, cab, msi files)
  • GZ
  • ZIP
  • RAR
  • binary PLIST
• database formats:
  • Redis database files
  • Ceph crush maps
• network protocol formats:
  • IRC
  • Pcap-NG
  • IP
  • Ethernet
  • PCAP
  • NTP
  • SNMP
  • TLS
  • TCP
  • UDP
  • DNS
• executable formats:
  • Portable executables (PE)
  • ELF
  • GameBoy ROM
• crypto related:
  • ASN.1
  • X.509 certificates
  • DER public and private keys
  • SSL/TLS packets
  • OpenPGP
• Programming Languages
  • Rust
  • Lua
  • Python
  • C
• interface definition formats:
  • Thrift
  • Protobuf
  • AIDL

[thehydroimpulse]

I'm writing a Thrift library for Rust that'll use Nom for both their IDL and the network protocol, so that can be another example (although in a different repo).

[Geal]

Nice idea, that will be useful! Please notify me when it is done, I will add a link in this list.

[filipegoncalves]

This looks interesting. Is anyone actively working on any of these parsers? I'd like to work on a few of these.

[Geal]

I have some code for a GIF one at https://github.com/Geal/gif.rs but it is hard to test, since the graphical tools in Piston change a lot.

You can pick any of them. Network packets may be the easiest, since they don't require a decompression phase.

I am using the gif example to see what kind of API can be built over nom. Most of the parsing example are done as one pass over the data, but often there is some logic on the side, and it is not easy to encode correctly.

[elij]

I've started a fastq (http://en.wikipedia.org/wiki/FASTQ_format) parser https://github.com/elij/fastq.rs

[Geal]

@elij this is a great idea! Was it easy to do?

[elij]

yup it's a great framework -- though I struggled a bit with eof so I borrowed some code from rust-config (https://github.com/elij/fastq.rs/blob/master/src/parser.rs#L69) -- is there a better solution?

[Geal]

yes, eof should be a parser provided by nom, I am just waiting for @filipegoncalves to send a PR 😉

[filipegoncalves]

Hah, sorry for my silence. I've been busy lately. I just sent a PR (#31).

I will be working on one of these example parsers as soon as I get some spare time. There are some great ideas in here!

[Keruspe]

I might give tar a try

[nelsonjchen]

Does this check off PCAP?

https://github.com/richo/pcapng-rs

[Geal]

pcap-ng and pcap are two different formats, right? It seems the consensus now is to move everything to pcap-ng, though.

[TechnoMancer]

I will try a FLAC parser, need to add quite a few things for it though.

[badboy]

ISO8601 is done in https://github.com/badboy/iso8601 (I hope it's mostly correct.)

[Geal]

ok, it should be up to date. More to come 😄