notesutils

Nice work, thanks for taking the time to write it up. I regret not doing that for my projects. I also did something similar back around 2014 in https://github.com/dunhamsteve/iwork but I didn't get much further than tables on the Numbers side before taking a break. There I translated iwork files to HTML. That code has been largely neglected since then, and I never wrote up my process. Like the other commenter, I based this on https://github.com/obriensp/iWorkFileFormat For ObjC programs that don't embed the descriptors, I wrote a python script that reverse engineers protobuf schemas from disassembled code: https://gist.github.com/dunhamsteve/224e26a7f56689c33cea4f0f... I don't remember what project that was for, but maybe it's useful to someone. And for Notes.app, I reverse engineered the description from the binary protobuf data. Since

I wrote python code to extract the notes years ago[1], but it's bit-rotted, so it no longer supports tables, raw stroke data, or the newer features like equations. Maybe I'll update it someday, but I'm not using notes at the moment and have too many other things going one. https://github.com/dunhamsteve/notesutils

Indeed, Apple Notes has a reasonably sophisticated CRDT, including support for tables: https://github.com/dunhamsteve/notesutils/blob/master/notes....

> some nice developer will make a little tool to take care of this There are already a bunch: - (discusses the internals) https://devlog.notespub.com/2022/08/site-generator-for-apple... - https://github.com/dunhamsteve/notesutils - https://github.com/ydkhatri/mac_apt - https://github.com/threeplanetssoftware/apple_cloud_notes_pa...

Last I checked it was not encrypted (unless you explicitly asked it to encrypt a note). Just gzipped protobuf data. I wrote an extractor about six years ago, maybe it'll help: https://github.com/dunhamsteve/notesutils It seems to still work for text. It looks like it doesn't dump tables anymore, so that bit may have changed (it was pretty convoluted). I think the extraction of drawing information doesn't work anymore, but they provide a fallback PNG file. I wanted it to be self-contained, so it has a little hand-rolled protobuf decoder and might be a little code-golfy.

I should write this stuff up, but I haven't. I do have some brief notes on the "Notes.app" format here: https://github.com/dunhamsteve/notesutils/blob/master/notes.... But I didn't discuss my methodology -- Generic decoding of protobuf, building up a schema as you go -- the tricky part there is that a byte array and a substructure look the same, so you have to try to decode it, and if successful, try that schema on the next example. Here is another fun technique - scanning through a disassembly of an Apple framework looking for assembly patterns that match the protobuf compiler output (this was dependent on which language was targeted by protobuf): https://gist.github.com/dunhamsteve/224e26a7f56689c33cea4f0f... So you find the serializer / deserializer code and figure out what the original protbuf spec looked like.

You did not miss it. There is some other, helpful, prior work in that subject by another author: https://github.com/dunhamsteve/notesutils/blob/master/notes....

Yeah, being able to get my data back out is a hard requirement for me too. It's one of the things that put me off of OneNote. A few years ago I wrote a few python scripts to export the Notes.app database to html and bear formats. I haven't really maintained them (although I still run it in a cronjob for backups), but I put them up on github in case it was useful to someone. https://github.com/dunhamsteve/notesutils

I haven't touched it in a while, but I wrote some code to extract data from apple notes and export to html, quiver, or bear formats. (bear is markdown-like, it looks like I targeted HTML for quiver.) It could be adapted to your needs if you know a little python: https://github.com/dunhamsteve/notesutils It might need updating for features added in the last few years, but the cron job I have it running in is still working. (The html version dumps the drawings as svg, but png versions are available too.)