GitJournal: Mobile first Markdown notes synchronized with Git
gitjournal.io
gitjournal.io
Too many app developers realize they need some kind of shared state "in the cloud" and end up with a database and a backend server. Many of them would be better off if they had a reliable way to push the state management to a user-controlled space, like a git repo or Dropbox
I used to store my DayOne journal in git, and boy, their cloud sync conflict resolution was sometimes quite destructive. Having the git history showed me that, and allowed me to correct it.
1. Decent desktop apps to sync the data, not everyone is comfortable hacking a Cron script. I've started with that - https://GitHub.com/GitJournal/git-auto-sync - but it's in the early stages.
2. For mobile apps, a way to drastically reduce the overhead. Cross compiling libgit2 + deps + NDK is a major pain, and jGit uses more modern java features so you need to target a higher version of Android. (This was in early 2019, when I first started GitJournal)
3. There is no competing with a simple http call vs the complexity of git. Plus the entire onboarding process.
4. Many many apps do write to Dropbox or Google Drive, but for me that's not a win as you aren't in control of your data.
5. While hosting your own git server is arguably easier than self hosting anything else, the majority of users in GitJournal (more than 80% use GitHub, this was about a year ago.) There is also a bias as people who self host are more likely to switch off the analytics. I don't send any message when analytics are switched off, so I don't know.
6. We need end-to-end encryption in Git, like keybase was providing, but open source and hostable in any git repo. I started a proof of concept of a custom remote where I just write all the git objects to another git repo, but haven't gotten very far.
Disclaimer: I'm the author of GitJournal.
I find this checks a ton of these boxes for me, and I find it really easy to get non-techies on board and actually be comfortable using it.
For locally shared documents, I sync them with Syncthing and edit them on Android in Markor, and it works pretty well.
I got different markdown editors on different devices and didn't want to use a public / cloud hosted git repo for my notes.
It works fine, as long you don't get in contact with iOS.
I got many devices and often let it sync my phone when I'm at home, because I'm struggling already enough with battery lifetime.
I also got it running on one of my dedicated servers, so I got an offsite backup, with systemd you can set it on autostart after an internet connection was established, so I don't really have to manage it anymore and I only see it basically in the logs of the packages that I update.
My only small tip for readers here is be careful treating any node as a safe backup.
A device that accidentally deletes everything in a folder will propagate out if you don't notice it happened.
But TBH in practice (as a very heavy user for my usecase) it's very rare, and a traditional backup system will cover you perfectly.
Also in upnotes we are using your local git command and I am planning to add encryption with your public key for few subfolders.
The first few minutes are the most important for an application. It should not fail and do as advertised. So far, this feels not fully tested or even UI tested. This would not lead to an easy upgrade to 'Pro'.
Note: removed statement about not intuitive of Git syncing. This is hidden in the hamburger menu instead of being part of the initial view and options.
Would you mind using the report a bug button in the sidebar? It'll create an email with some debugging info.
I thought I'd fixed the issue, but the fix is still pending review on iOS. However, if you're Android, you should have it.
I’ve requested a refund. But please don’t take that as discouragement. I want this workflow (editing markdown notes synced to git) very much. I’ve tried various ways to hack it, like connecting 1Writer (editor) to Working Copy (git client). Although both apps are very slick, the integration is not.
Editor apps are also reluctant to provide git integration for the reasons you listed, plus it will reduce future revenue from a hosted solution they could offer. I’m not too optimistic that there will be many apps in this space. That’s good news for you!
Best of luck!
My apologies, I introduced a bug and didn't expect this to get posted on HNs. The fix is pending review on iOS, so hopefully you should receive it soon. (Version 1.85.1 has the fix)
Regardless, I'd love it if you could reach out to me (vhanda at gitjournal.io) and explain your workflows, additionally, I'll be happy to provide you some coupon codes so you can try the basic version of GitJournal without paying. Especially considering your horrid experience.
Once the bugs are fixed, I look forward to trying this out. The auth-flow with GitHub was quite good, until the error at the end :)
Best of luck with the project.
I tested it properly this time. Version 1.85.2, I'm just uploading it for review.
If you want structured data it's pretty good IMHO. Take a look: https://developers.verida.io/docs/tutorial/databases
If you feel strongly about it, please consider contacting the purveyors of your favorite software and ask why they don't have Git integration.
(Edit: libgit2, not libgit.)
The typical scenario for me is start typing something in and then get interrupted (notification, time based appointment, someone needing my attention), when I get back to entering I might be at a different machine (like started on my phone, got interrupted, then sitting at my computer and remember to finish).
I don't see how git sync helps this issue.
In many situations, it's better to version separate pages/items/notes separately, and have a conflict to a single note leave the rest of your repo working and syncing.
It's not perfect, but in the worst case, the wrong option is chosen, but no data is lost since the history is there in Git.
It’s fully seamless, private, and free.
I do agree that a standardized sync protocol and file format that could be pointed at any arbitrary server would be terrific. I just disagree that git is that format for many use cases.
I almost never truly want to delete any note. And most changes are additive so don’t take much extra storage due to versioning.
I find it maddening to use a note app that doesn’t have detailed change logs and git’s history and blame is really useful.
I've been using a single git repo to store notes for years. And my entire repo is a few MB.
$ du -sh ~/Notes/
47M /home/dotancohen/Notes/
$ du -sh ~/Notes/.git
21M /home/dotancohen/Notes/.git
$ git -C ~/Notes/.git gc
$ du -sh ~/Notes/
44M /home/dotancohen/Notes/Essentially, a bare git instance.
I have the git syncing tools but most apps dont save to flat file and the ones that do tend to require hitting export -> backup.
This is usually the premium offering( eg: obsidian) in their product.
Sorry about that. A bugfix version will hopefully be soon available.
I'm happy to answer any questions.
Otherwise, feedback about GitJournal is also greatly appreciated.
Since the history is still preserved, no data is lost.
- I commit and pull + push each time a note is modified / created / deleted. And whenever you pull to refresh or tap the sync button. Optionally, this can be disabled, and it will only pull+push when you press the button.
- I only stay on the main branch, the user has an option to switch branches. I have no plans and no requests / use-cases for doing anything more with branches.
The website is built with Hugo [0], so it should be easy enough.
My initial (and even now) aim with GitJournal was to satisfy 2 of my needs - A good journalling app where I have control of my data. I used to use Journey [0], but they kept adding more and more cruft, and having to export my data + relying on Google wasn't something I liked. The other is Google Keep. I still have a long way to go to get their level of usability though. Things just feel so much nicer in Google Keep.
Now, since I've been using Obsidian more, I'm starting to incorporate more features from it. So UI is a mix of all 3.
I just noticed that the screenshot carousel in the website is broken.
- [0] https://journey.cloud/
Thank you for developing this app and for being so responsive to support requests Vishesh!
One small discrepancy between the 2 products is that GitJournal handles the file’s title as both the filename and an h1 inside it, while Obsidian doesn’t sync both by default IIRC. I’m using an obsidian plugin to keep them in sync (i.e. overwrite one when the other changes). I think GitJournal has the correct default behavior but with how popular obsidian is, a setting to behave like obsidian might be useful. My 2 cents.
Regarding stability, there sadly are some crashes on ios which I haven't managed to figure out at all. I'm planning a big move next week where I bump the version of libgit2, since I'm lagging behind by almost 1.5 years now, and I want to make it a regular occurrence. I'm hoping that will help with the stability.
The next biggest wish I have is for inclusion and management of binary files. Photos, videos, etc. It'd be nice for them to not be in the git repo, but to still be distributed, addressable, annotatable, and with some kind of cross-platform UI (Linux, Windows, Mobile, Web).
Even better if it's easy to automate workflows to publish to a public personal wiki / blog / website.
I do, however, want to support partial clones and easier tools to cleanup the history and reduce the repo size.
libgit2 already supports filters and that’s all you need to build lfs support (actually, you don’t even really need to use them if you don’t want to). Lfs itself is not a feature of git. It’s a specification that defines a protocol for replacing large files with pointers (via the use of the smudge and clean filters) and uploading said large files to another location.
Source: I am the cofounder and former CTO of forestry.io, a git backed content management system, and I implemented lfs support in conjunction with libgit2 in that product about 4 years ago.
I'm not super familiar with LFS beyond its name and stated intentions, but after a cursory review it seems to be extremely manual/tedious, so much that it makes git itself look like a paragon of automated magic and user-friendliness. Maybe that's an unfair assessment, or maybe it could be made automatic, but I'm less sure that its the solution I'd want.
The use case I'm interested in is similar to the comment above "binary files. Photos, videos, etc", What I was hoping for with LFS would be like a thin clone of all my personal files where some files are only present in name with their contents downloaded on-demand on some devices. So I'm not sure if history pruning would help my use-case much.
The app seems nice but I think git history and durability is a good feature.
They have built the mobile apps for you as a convenience. But if that's not a good fit for you, The project is open source[1]. Build it yourself.
What I'm pushing for is open formats + git for sync. That way, you can easily combine different tools. GitJournal is never going to do everything, the note taking app world is way too complex.
Instead, my plan was for GitJournal to be one other tool in your workflow.
I write "fairly happy", because there is a longstanding issue with the Android app where notes that have more than 10 lines start to experience crazy lag when editing. So I mainly use the desktop version for writing.
Wish they would fix the Android bug!
Also, the entire source code (pro features as well) are completely open source.
Regardless, I'm not happy that you feel cheated. Is there someway to ask Apple for a refund?
Edit: it isn’t free on the apple app store. I failed to realize that I t shows as free for me because I already purchased it of course.
Maybe it's cause I had earlier experimented with a subscription instead of paying once for the new features.
Shoot me an email please, I'll provide you with some coupon codes so you can get the pro features without paying, just in case you want to try it out again.
How does this hold up with storing notes in Git (which usually means syncing to github etc.)? I mean of course you can self-host to keep your notes privately, but does this support encrypted notes? I imagine git history works poorly if notes are encrypted.
I'm working on an end-to-end encrypted git solution, but it's still in the prototype phase.
https://github.com/mozilla/sops
https://github.com/android-password-store/Android-Password-S...
Both are extremely useful secrets oriented git tools with support for things like PGP encryption. Both will encrypt with multiple keys too, making sharing relatively easy. The android pass app even manages SSH keys for pushing and pulling. There may be good inspiration in those repos, or even code you can borrow.
Also, thanks so much for making this: it is elegant and lovely. Keep it up!
Internally, the idea is to just encrypt all the git objects, one per file, and store those in a git repo. It could also be stored in S3, as storing it in a repo doesn't help, but Git Repos have the least vendor lock-in.
interval_in_seconds=0.5
previous=""
while true;
do
x=$(ls -R -al ./content | md5);
if [[ "$x" != "$previous" ]]; then
previous=$x
git add *
git commit -m "Content update"
# ...
fi;
sleep $interval_in_seconds;
done
Of course it lacks a lot of the functionality that GitJournal has. It is just meant to keep a history of changes.You can basically call `git-auto-sync daemon add`, and it will register itself as a systemd user process (on Linux, other OSs are also supported), and monitor that folder for changes + poll every x minutes.
It's still in the early stages, but I'd love to get your feedback.
Edit: Or you can just call `git-auto-sync sync` to commit + pull / rebase + push.
More info from National Treasure Julia Evans: https://jvns.ca/blog/2020/06/28/entr/
Something like this:
git ls-files | entr autocommit.shUnfortunately this app doesn't hold a candle to Markor when it comes to editing and navigating. - Navigating links between files is a pro feature - There is no markdown syntax highlighting while editing - There is no way to favorite or bookmark files to easily navigate to them
As someone who uses GitHub and Markdown as a personal wiki this app doesn't have the features required to be usable for me.
Looks like MGit will continue to be an unfortunate part of my life.
If you had to pick one feature, which I could implement to help, which would it be?
Disclaimer: GitJournal author
Personally i have these things that bother me:
* an option for a fixed width font in markdown editor (i like the look of the raw editor but i like the convenience of the markdown one)
* code syntax highlighting
* creating/changing tables with less effort
* very aggressive quoted text styling in light mode
Thanks for making reasonable software. :)
I have always used the builtin wiki from Github/Gitlab to achieve similar note taking. But attachments and links (to and between folders) are a pain to get right ;) So I'd be interested in an app that helps there.
Would love if https://github.com/GitJournal/GitJournal/issues/352 were prioritized, though. Making adding new repos easier would be a big deal for my workflow, as I have multiple customers that all need their own repos, sometimes more than one.
In terms of polish, it's comparable to Craft, but with only the feature-set of Markdown. A few things are missing such as tables and backlinks.
It uses an SQL database for performance reasons, but export to Obsidian is possible within minutes. Indeed, while other file-based alternatives such as NotePlan fail to deal with my 5000+ notes, including images and attachments, Bear is still as smooth as on day one.
Just don't expect new features any time soon. The Bear team has been working on the next version for years, with no end in sight.
Scary obligatory warning: Panda is alpha software (and probably not something that you should trust with your notes). The table support in Panda is nice though.
I switched to Zettel Notes which has been reliable and fast
I've now reverted back to libgit2, since I don't need to access it via Dart -> Flutter Message -> Android -> NDK -> custom C wrapper -> libgit2.
The monetisation options are incredibly user-respecting too. Although I don't need the Pro features, I'm on the monthly subscription just to support the project.
Also, you might want to look at this bug as it might be affecting your cash flow: https://github.com/GitJournal/GitJournal/issues/612
Thanks for pointing the bug out!
https://www.spiceworks.com/it-security/cyber-risk-management...
If you're a student or having financial problems, please let me know, I'll be happy to generate some coupon codes.for you.
I tried experimenting with running a background async job every 'x' minutes, but encountered problems with Flutter + background tasks so I put it on hold.
The updates are auto-synced with the Android updates, so you'll always get them at the same time or sooner.
For my own notes, I've mostly been using Markdown, which doesn't have fenced code blocks, but I've been thinking I might upgrade to a different language like CommonMark or GitLab-Flavored Markdown that does support them. One of the main reasons is the possibility of embedding non-text things in my notes; for example, the other day I was messing around with exact pole-zero cancellation in DSP in https://nbviewer.org/url/canonical.org/~kragen/sw/dev3/pole-..., which involved a lot of TeX equations and data plots of algorithmically generated output. Jupyter notebooks can do those things, but they have a number of extremely serious drawbacks for this sort of thing, such as irreproducibility, aggressively backwards-incompatible file formats, lack of interactivity (despite ipywidgets), unacceptably large "source code" file size, and spurious merge conflicts in Git.
An interesting approach to solving this problem is Mermaid (https://docs.gitlab.com/ee/user/markdown.html#diagrams-and-f...), which lets you include a ```mermaid fenced code block in your Markdown source and replaces it with a rendered image of a boxes-and-arrows diagram in the HTML output. This is sort of backwards-compatible with regular GLFM because things that don't support Mermaid just show the diagram's source code. Mermaid's "flowchart" diagram type is vaguely similar to Graphviz, but I think Graphviz has better syntax and nicer output, so I'd rather use Graphviz for this; Mermaid also has Gantt charts, UML diagrams, sequence diagrams, and entity-relationship diagrams.
GitLab also supports language-tagged fenced code blocks for "math" (KaTeX's subset of LaTeX), "plantuml" (disabled by default, does UML diagrams), and Kroki (also disabled by default, supports Graphviz, Ditaa (warning, domain stolen by SEOs) for flowcharts, Wavedrom for digital timing diagrams, Vega-Lite for data plotting with interactive exploration from JSON, Svgbob for a fairly comprehensive set of ASCII-art line drawings, Burntsushi's Erd for entity-relationship diagrams, bytefield-svg, Pikchr for PIC-like things, several different ASCII-to-UML things, and another dozen or so other similar things).
Pikchr, from the Fossil/SQLite folks, is also designed for embedding in GFML and similar Markdown-derived languages, even without Kroki.
But most of these are just data formats, rather than rich data models or executable code like I'd need for the DSP algorithms I mentioned above. The closest thing I've found for that is Yihui Xie's R Notebooks: https://bookdown.org/yihui/rmarkdown/notebook.html where you can put fenced R code blocks into your Markdown and display the output inline both in the RStudio IDE and in the final rendered document. This still has Jupyter's problem with reproducibility, but solves the problems with massive source files, file-format backwards compatibility, and spurious merge conflicts.
However, R Notebooks require you to write your code in R, which isn't ideal for all purposes, and as far as I know the rendered results don't have a way to do Vega-style or ObservableHQ-style interactive exploration. Also, pretty often I find it easier to sketch a diagram with a pencil than to write code to produce it.
And, for things like analog circuit schematics or 3-D models, while a compact textual representation is achievable, it often isn't the ideal user interface for editing. Mermaid Live, OpenSCAD, and dot -Tx11 filename.gv are three approaches where the text-to-picture flow is one-way, and while they're pretty great, you still spend a fair bit of time hunting through source code for the thing you want to edit and trying to diagnose syntax errors.
So I'd like to have a note-taking system that supports Git and Commonmark, reproducibility so that I don't need to check in giant binary files representing render outputs, plug-in visualizers for different kinds of quasi-textual content, and plug-in editors for editing things like 3-D models, circuit schematics, and 2-D sketches. And I'd like to be able to do things like run a SPICE simulation of a circuit schematic, a FEM simulation of a 3-D model, or a numerical integration of a Modelica model.
Is there anything like that out there? What are the alternative approaches to this kind of algorithmic exploration such as ObservableHQ?
The DSP notebook I mentioned above also includes an input audio sample and several processed versions of it, which you can't hear in the online Jupyter notebook viewer but you can hear if you run it locally. The input audio sample would pretty much have to be an external file, same as a photo, but ideally the processed versions could be recomputed reproducibly rather than stored in Git and copied around all over the net.
disclosure: I’m the author of Plain Org.
There is basic support because of another open source org mode app - Orgro (https://orgro.org), - the author was even gracious enough to relicense his code away from GPL, so that I could use it. since GitJournal is shipped on the iOS store and GPL would not be compatible.
However, GitJournal internally expects everything to be markdown + yaml. So anything else are hacks I've done on top. At some point, I want to refactor and remove this hard dependency on Markdown.
Edit: I mistakenly thought the app was Orgzy. I've updated the comment. Sorry.
I mean I respect your decision to make it open-source or not, but why does the store matter ?
[0] https://opensource.stackexchange.com/questions/9500/is-apple...
[1] https://news.ycombinator.com/item?id=12827624
[2] https://github.com/KDE/kdeconnect-ios/blob/master/License.md
The relevant libraries are https://github.com/amake/org_flutter and https://github.com/amake/org_parser.