cljoly/hmrc2ledger

Disclaimer: This is not financial advice. It may be outdated or wrong, do your own research.

Convert CSV to ledger price db, to compute foreign transaction CGT (and section 104 holdings).

If you are using Ledger or one of its variants for your plain text accounting, you get to manage your finances a bit more like you manage code. With plain text files, in your favorite editor.

And yet, as opposed to code in most languages, you don’t have a Language Server Protocol implementation. So it’s a bit harder to get completions on accounts, tags, payees… You have to rely on bespoken support from your editor and that’s often imperfect or slow. Plus, if you declare your accounts and payees, you also can’t jump to the definition easily.

Enter ctags configuration for ledger, which this project provides. It generates tag file that’s understood by your editor, often out of the box (vim/neovim, emacs) or with a generic plugin (Atom, VS Code, TextMate and more). Congrats, your editor now has very fast completions and code navigation!

Installation

You need to have ctags installed¹.

Then drop ledger.ctags in ~/.config/ctags (or whatever ctags is configured to read).

Usage

ctags my/ledger/journal.ldg

will collect all the elements formally declared in the journal file. This means that in the following file:

tag UUID
account Assets
payee Shop
commodity EUR

2022-01-01 * Someone
    Expenses:Food   $10
    Liabilities:Credit Card

only UUID, Assets, Shop and EUR are detected, because they are the only one declared and so the only ones with a definition to jump to. Someone, Expenses, Expenses:Food, $, Liabilities and Liabilities:Credit Card are not declared, so they are ignored by ctags.

Initial List of Accounts, Payees…

As mentioned, ctags only collects elements that are declared. You could write the declarations manually, if you want to have fine-grained control of which accounts you want to actively use and have completions for. But to ease initial bootstrapping, you may want to start by adding everything you currently use in your ledger journals. You can do that with a couple commands.

For accounts, you can use this bash snippet:

ledger accounts | awk '{ print "account", $0}'

For payees:

ledger payees | awk '{ print "payee", $0}'

For tags:

ledger tags | awk '{ print "tag", $0}'

The snippets above output proper ledger definitions for accounts, payees and tags. You can insert these outputs in your journal or in a separate file with all your declarations (say declarations.ledger), that you can then include in your main journal file.

Don’t forget that ctags does not follow includes, so you should pass the file with the definitions to ctags (e.g. ctags declarations.ledger)

Multiple files

If you have multiple ledger journal files, pass them all to ctags (e.g. ctags *.ledger *.ldg) or run it on the current directory:

ctags -R .

Automatic Update with Chezmoi

Chezmoi external feature allows you to auto update and manage the ledger.ctags file, by adding the following to your ~/.local/share/chezmoi/.chezmoiexternal.toml:

[".config/ctags/"]
    type = "archive"
    include = [ "*/ledger.ctags" ]
    url = "https://github.com/cljoly/ledger.ctags/archive/master.zip/"
    stripComponents = 1

Contributing

Contributions (documentation or code improvements in particular) are welcome, see contributing!

Feel free to fill an issue if your declared accounts, tags, payee, alias… are not properly detected.

Acknowledgments

I would like to thank the creators, and all the contributors, of ctags and ledger. This is small configuration to make these two great projects work better together.

into-ledger helps categorization of CSV transactions and conversion into ledger format for consumption by ledger-cli.org. It makes importing hundreds of transactions into ledger a breeze. I typically get close to a hundred transactions per account per month myself, which is why I wrote this tool.

Features

• Accurate : Uses a much more accurate tf-idf expense classifier than used by cantino/reckon.
• Includes and Aliases : Correctly parses your existing journal file, handling all includes and account aliases.
• Keyboard Shortcuts : Assigns dynamic keyboard shortcuts, so classifying transactions is just a keystroke away.
• Auto save : Uses temporary storage (boltdb) to persist transactions that you have categorized or acknowledged to be correctly categorized, so you can quit whenever you want, without the risk of losing the work done so far.
• Deduplication : Deduplicates incoming transactions from CSV against the transactions already present in ledger journal. This allows an easy resume from a broken workflow.
• Nice UI : Colors and formatting, because it’s not just about getting things done. It’s also about making them look nice!

This fork adds:

• Fixes: various improvement, in particular to the CSV parsing
• Fuzzy selection: with fzf
• More documentation: admittedly still in progress
• More modern, cleaner codebase: this fork vs upstream
• Support of payee aliases: file with internal aliases TODO

This fork removes:

• Plaid support: with better CSV parsing, the tool is more generic and can use data from many more websites

Install

Binaries

If you understand the security implications, grab a binary from the release page and run it.

Build it yourself

A better option, security-wise, is to install the go toolchain and run

go get -u joly.pw/into-ledger

Help

Usage of ./into-ledger:
  -a string
    	Name of bank account transactions belong to.
  -c string
    	Set currency if any.
  -conf string
    	Config directory to store various into-ledger configs in. (default "/home/mrjn/.into-ledger")
  -comma string
	Separator of fields in csv file
  -csv string
    	File path of CSV file containing new transactions.
  -d string
    	Express your date format in numeric form w.r.t. Jan 02, 2006, separated by slashes (/). See: https://golang.org/pkg/time/ (default "01/02/2006")
  -debug
    	Additional debug information if set.
  -ic string
    	Comma separated list of columns to ignore in CSV.
  -j string
    	Existing journal to learn from.
  -o string
    	Journal file to write to. (default "out.ldg")
  -opt string
	Extra option to pass to ledger commands
  -s int
    	Number of header lines in CSV to skip

Usage

# Importing from Citibank Australia
$ into-ledger -j ~/ledger/journal.ldg -csv ~/ledger/ACCT_464_25_07_2016.csv --ic "3,4" -o out.data -a citi -c AUD -d "02/01/2006"

# Importing from Chase USA. Skips the first line in CSV file. Also skips the first (0) and second (1) column in csv. Outputs to out.data. Sets currency as USD.
$ into-ledger -j ~/ledger/journal.ldg -csv ~/ledger/Activity.CSV --ic "0,1" -o out.data -a chase -c USD -s 1

Having to specify these command line arguments over and over again is annoying. So, instead you can create a config file in “$HOME/.into-ledger/config.yaml”, storing the flag values for reuse, like so:

accounts:
  chase:
    c: USD
    j: /home/mrjn/ledger/journal.ldg
    d: 01/02/2006
    ic: "0,1"
    o: /home/mrjn/ledger/chase.out
    s: 1
  cba-smart:
    c: AUD
    j: /home/mrjn/ledger/journal.ldg
    d: 02/01/2006
    ic: "3"
    o: /home/mrjn/ledger/cba.out

Note: The way config is stored has changed recently. Please update your version of into-ledger using go get -u -v github.com/manishrjain/into-ledger. Also, update your config file.

Now you can just run: into-ledger -a chase -csv <input-csv>, or into-ledger -a cba-smart -csv <input-csv>

Dates

into-ledger requires you to specify the date format in numeric form w.r.t. Jan 02, 2006. This is how Go language parses dates. This is frequently a cause of confusion among folks unfamiliar with the language. So, please find here the commonly used date formats and how to specify them in into-ledger.

Keyboard Shortcuts

One of the great advantages of using into-ledger is how quickly you can categorize a transaction. Most of the times the underlying categorization algorithm is smart enough to do the right thing for you. However, for the rest, into-ledger shows you keyboard shortcuts to pick the right category.

into-ledger uses a keys module I wrote, which automatically assigns shortcuts to categories and persists them in ~/.into-ledger/shortcuts.yaml. However, you might want to use certain keys for certain categories. In that case, feel free to hand-edit the shortcuts.yaml file. Just ensure that the same shortcut isn’t being used twice in the file.

Tip: If you want to assign a shortcut to a category, but it’s being used by another category, feel free to delete that category block from the shortcuts file. into-ledger will automatically reassign a new shortcut to the deleted category, and write it back.

Screenshots

Parse transactions from CSV, and show automatically picked categories to be reviewed.

Detect duplicates transactions in CSV, which are already present in ledger journal.

Categorize transaction using persistent and dynamic keyboard shortcuts.