template: Template amounts are now written with fuller expressions.
This makes it possible to support transactions that divide funds in ways other than a simple percentage split. For example, there might be tax withheld, or a flat fee expense on each transaction.
This commit is contained in:
Brett Smith
parent
cebd1481ec
commit
3b821cbbee
8 changed files with 231 additions and 96 deletions
81
README.rst
81
README.rst
|
|
@ -24,15 +24,15 @@ Writing templates
|
|||
A template looks like a Ledger entry with a couple of differences:
|
||||
|
||||
* You don't write the first payee line.
|
||||
* You allocate percentages to each account, rather than specific currency amounts.
|
||||
* Instead of writing specific currency amounts, you write simple expressions to calculate amounts.
|
||||
|
||||
Here's a simple template for Patreon patron payments::
|
||||
|
||||
[DEFAULT]
|
||||
template patreon income =
|
||||
;Tag: Value
|
||||
Income:Patreon -100%%
|
||||
Accrued:Accounts Receivable:Patreon 100%%
|
||||
Income:Patreon -{amount}
|
||||
Accrued:Accounts Receivable:Patreon {amount}
|
||||
|
||||
Let's walk through this line by line.
|
||||
|
||||
|
|
@ -42,12 +42,41 @@ Every setting in your configuration file has to be in a section. ``[DEFAULT]``
|
|||
|
||||
The first line of the template is a Ledger tag. The program will leave all kinds of tags and Ledger comments alone, except to indent them nicely.
|
||||
|
||||
The next two lines split the money across accounts. They follow almost the same format as they do in Ledger: there's an account named, followed by a tab or two or more spaces, and then an amount. However, rather than a currency amount, specify a percentage amount between -100% and 100%. You can specify any number of splits you want, as long as the negative splits total -100% and the positive splits total 100%. ``%`` is a special character in INI file syntax; writing ``%%`` is the way to get a literal ``%`` in the string. Alternatively, you can omit the ``%`` altogether; import2ledger will still treat any floating point value at the end of the line as a percentage.
|
||||
The next two lines split the money across accounts. They follow almost the same format as they do in Ledger: there's an account named, followed by a tab or two or more spaces, and then an expression. Each time import2ledger generates an entry, it will evaluate this expression using the data it imported to calculate the actual currency amount to write. Your expression can use numbers, basic arithmetic operators (including parentheses for grouping), and imported data referred to as ``{variable_name}``.
|
||||
|
||||
If the amount of currency being imported doesn't split evenly, spare change will be allocated to the last split to keep the entry balanced.
|
||||
import2ledger uses decimal math to calculate each amount, and rounds to the number of digits appropriate for that currency. If the amount of currency being imported doesn't split evenly, spare change will be allocated to the last split to keep the entry balanced on both sides.
|
||||
|
||||
Refer to the `Python documentation for INI file structure <https://docs.python.org/3/library/configparser.html#supported-ini-file-structure>`_ for full details of the syntax. Note that import2ledger doesn't use ``;`` as a comment prefix, since that's the primary comment prefix in Ledger.
|
||||
|
||||
Template variables
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
import2ledger templates have access to a few variables for each transaction that can be included anywhere in the entry, not just amount expressions. In other parts of the entry, they're treated as strings, and you can control their formatting using `Python's format string syntax <https://docs.python.org/3/library/string.html#formatstrings>`_. For example, this template sets different payees for each side of the transaction::
|
||||
|
||||
template patreon income =
|
||||
Income:Patreon -{amount}
|
||||
;Payee: {payee}
|
||||
Accrued:Accounts Receivable:Patreon {amount}
|
||||
;Payee: Patreon
|
||||
|
||||
Every template can use the following variables:
|
||||
|
||||
================== ==========================================================
|
||||
Name Contents
|
||||
================== ==========================================================
|
||||
amount The total amount of the transaction, as a simple decimal
|
||||
number (not currency-formatted)
|
||||
------------------ ----------------------------------------------------------
|
||||
currency The three-letter code for the transaction currency
|
||||
------------------ ----------------------------------------------------------
|
||||
date The date of the transaction, in your configured output
|
||||
format
|
||||
------------------ ----------------------------------------------------------
|
||||
payee The name of the transaction payee
|
||||
================== ==========================================================
|
||||
|
||||
Specific importers and hooks may provide additional variables.
|
||||
|
||||
Supported templates
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -70,40 +99,14 @@ Patreon
|
|||
|
||||
This template can use these variables:
|
||||
|
||||
``country_name``
|
||||
The full name of the country VAT was withheld for
|
||||
|
||||
``country_code``
|
||||
The two-letter ISO country code of the country VAT was withheld for
|
||||
|
||||
Template variables
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
import2ledger templates have access to a few variables for each transaction that can be included in the output entry using `Python's format string syntax <https://docs.python.org/3/library/string.html#formatstrings>`_. For most uses, all you need to do is write ``{variable_name}`` in the template, and the variable's value will be filled in there. For example, this template sets different payees for each side of the transaction::
|
||||
|
||||
template patreon income =
|
||||
Income:Patreon -100%%
|
||||
;Payee: {payee}
|
||||
Accrued:Accounts Receivable:Patreon 100%%
|
||||
;Payee: Patreon
|
||||
|
||||
You can use the following variables:
|
||||
|
||||
================== ==========================================================
|
||||
============== ============================================================
|
||||
Name Contents
|
||||
================== ==========================================================
|
||||
amount The total amount of the transaction, as a simple decimal
|
||||
number (not currency-formatted)
|
||||
------------------ ----------------------------------------------------------
|
||||
currency The three-letter code for the transaction currency
|
||||
------------------ ----------------------------------------------------------
|
||||
date The date of the transaction, in your configured output
|
||||
format
|
||||
------------------ ----------------------------------------------------------
|
||||
payee The name of the transaction payee
|
||||
================== ==========================================================
|
||||
|
||||
Custom hooks may add more template variables.
|
||||
============== ============================================================
|
||||
country_name The full name of the country VAT was withheld for
|
||||
-------------- ------------------------------------------------------------
|
||||
country_code The two-letter ISO country code of the country VAT was
|
||||
withheld for
|
||||
============== ============================================================
|
||||
|
||||
Other output options
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -112,6 +115,8 @@ Various options control how import2ledger formats dates and currency amounts. R
|
|||
|
||||
date_format = %%Y-%%m-%%d
|
||||
|
||||
(``%`` is a special character in the configuration file syntax. ``%%`` represents a literal percent sign.)
|
||||
|
||||
Configuration sections
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
|
|||
|
|
@ -241,6 +241,7 @@ class Configuration:
|
|||
signed_currencies=[code.strip().upper() for code in section_config['signed_currencies'].split(',')],
|
||||
signed_currency_fmt=section_config['signed_currency_format'],
|
||||
unsigned_currency_fmt=section_config['unsigned_currency_format'],
|
||||
template_name=config_key,
|
||||
)
|
||||
|
||||
def setup_logger(self, logger, section_name=None):
|
||||
|
|
|
|||
|
|
@ -1,49 +1,153 @@
|
|||
import collections
|
||||
import datetime
|
||||
import decimal
|
||||
import functools
|
||||
import io
|
||||
import operator
|
||||
import re
|
||||
import tokenize
|
||||
|
||||
import babel.numbers
|
||||
|
||||
class AccountSplitter:
|
||||
Split = collections.namedtuple('Split', ('account', 'percentage'))
|
||||
from . import errors
|
||||
|
||||
def __init__(self, sign, signed_currencies, signed_currency_fmt, unsigned_currency_fmt):
|
||||
self.splits = []
|
||||
self.sign = sign
|
||||
class TokenTransformer:
|
||||
def __init__(self, source):
|
||||
try:
|
||||
source = source.readline
|
||||
except AttributeError:
|
||||
pass
|
||||
self.in_tokens = tokenize.tokenize(source)
|
||||
|
||||
@classmethod
|
||||
def from_bytes(cls, b):
|
||||
return cls(io.BytesIO(b).readline)
|
||||
|
||||
@classmethod
|
||||
def from_str(cls, s, encoding='utf-8'):
|
||||
return cls.from_bytes(s.encode(encoding))
|
||||
|
||||
def __iter__(self):
|
||||
for ttype, tvalue, _, _, _ in self.in_tokens:
|
||||
try:
|
||||
transformer = getattr(self, 'transform_' + tokenize.tok_name[ttype])
|
||||
except AttributeError:
|
||||
raise ValueError("{} token {!r} not supported".format(ttype, tvalue))
|
||||
yield from transformer(ttype, tvalue)
|
||||
|
||||
def _noop_transformer(self, ttype, tvalue):
|
||||
yield (ttype, tvalue)
|
||||
|
||||
transform_ENDMARKER = _noop_transformer
|
||||
|
||||
def transform_ENCODING(self, ttype, tvalue):
|
||||
self.in_encoding = tvalue
|
||||
return self._noop_transformer(ttype, tvalue)
|
||||
|
||||
def transform(self):
|
||||
out_bytes = tokenize.untokenize(self)
|
||||
return out_bytes.decode(self.in_encoding)
|
||||
|
||||
|
||||
class AmountTokenTransformer(TokenTransformer):
|
||||
SUPPORTED_OPS = frozenset('+-*/()')
|
||||
|
||||
transform_NAME = TokenTransformer._noop_transformer
|
||||
|
||||
def transform_NUMBER(self, ttype, tvalue):
|
||||
yield (tokenize.NAME, 'Decimal')
|
||||
yield (tokenize.OP, '(')
|
||||
yield (tokenize.STRING, repr(tvalue))
|
||||
yield (tokenize.OP, ')')
|
||||
|
||||
def transform_OP(self, ttype, tvalue):
|
||||
if tvalue == '{':
|
||||
try:
|
||||
name_type, name_value, _, _, _ = next(self.in_tokens)
|
||||
close_type, close_value, _, _, _ = next(self.in_tokens)
|
||||
if (name_type != tokenize.NAME
|
||||
or close_type != tokenize.OP
|
||||
or close_value != '}'):
|
||||
raise ValueError()
|
||||
except (StopIteration, ValueError):
|
||||
raise ValueError("opening { does not name variable")
|
||||
yield (tokenize.NAME, name_value)
|
||||
elif tvalue in self.SUPPORTED_OPS:
|
||||
yield from self._noop_transformer(ttype, tvalue)
|
||||
else:
|
||||
raise ValueError("unsupported operator {!r}".format(tvalue))
|
||||
|
||||
|
||||
class AccountSplitter:
|
||||
def __init__(self, signed_currencies, signed_currency_fmt, unsigned_currency_fmt,
|
||||
template_name):
|
||||
self.splits = collections.OrderedDict()
|
||||
self.signed_currency_fmt = signed_currency_fmt
|
||||
self.unsigned_currency_fmt = unsigned_currency_fmt
|
||||
self.signed_currencies = set(signed_currencies)
|
||||
self.unsplit = sign * 100
|
||||
self.template_name = template_name
|
||||
self._last_template_vars = object()
|
||||
|
||||
def add(self, account, percentage_s):
|
||||
int_pctage = decimal.Decimal(percentage_s.rstrip('%'))
|
||||
self.unsplit -= int_pctage
|
||||
percentage = None if self.unsplit == 0 else (int_pctage / 100)
|
||||
self.splits.append(self.Split(account, percentage))
|
||||
def add(self, account, amount_expr):
|
||||
try:
|
||||
clean_expr = AmountTokenTransformer.from_str(amount_expr).transform()
|
||||
except ValueError as error:
|
||||
raise errors.UserInputConfigurationError(error.args[0], amount_expr)
|
||||
self.splits[account] = compile(clean_expr, self.template_name, 'eval')
|
||||
|
||||
def _currency_decimal(self, amount, currency):
|
||||
return decimal.Decimal(babel.numbers.format_currency(amount, currency, '###0.###'))
|
||||
|
||||
def _balance_amounts(self, amounts, to_amount):
|
||||
cmp_func = operator.lt if to_amount > 0 else operator.gt
|
||||
should_balance = functools.partial(cmp_func, 0)
|
||||
remainder = to_amount
|
||||
to_account = None
|
||||
for account, amount in amounts.items():
|
||||
if should_balance(amount):
|
||||
remainder -= amount
|
||||
to_account = account
|
||||
if to_account is None:
|
||||
pass
|
||||
elif (abs(remainder) / abs(to_amount)) >= decimal.Decimal('.1'):
|
||||
raise errors.UserInputConfigurationError(
|
||||
"template can't balance amounts to {}".format(to_amount),
|
||||
self.template_name,
|
||||
)
|
||||
else:
|
||||
amounts[to_account] += remainder
|
||||
|
||||
def _build_amounts(self, template_vars):
|
||||
amount_vars = {k: v for k, v in template_vars.items() if isinstance(v, decimal.Decimal)}
|
||||
amount_vars['Decimal'] = decimal.Decimal
|
||||
amounts = collections.OrderedDict(
|
||||
(account, self._currency_decimal(eval(amount_expr, amount_vars),
|
||||
template_vars['currency']))
|
||||
for account, amount_expr in self.splits.items()
|
||||
)
|
||||
self._balance_amounts(amounts, template_vars['amount'])
|
||||
self._balance_amounts(amounts, -template_vars['amount'])
|
||||
return amounts
|
||||
|
||||
def _iter_splits(self, template_vars):
|
||||
amounts = self._build_amounts(template_vars)
|
||||
if template_vars['currency'] in self.signed_currencies:
|
||||
amt_fmt = self.signed_currency_fmt
|
||||
else:
|
||||
amt_fmt = self.unsigned_currency_fmt
|
||||
for account, amount in amounts.items():
|
||||
amt_s = babel.numbers.format_currency(amount, template_vars['currency'], amt_fmt)
|
||||
yield ' {:45} {:>19}\n'.format(account, amt_s)
|
||||
|
||||
def render_next(self, template_vars):
|
||||
try:
|
||||
account, percentage = next(self._split_iter)
|
||||
except (AttributeError, StopIteration):
|
||||
self._split_iter = iter(self.splits)
|
||||
account, percentage = next(self._split_iter)
|
||||
self._remainder = self.sign * template_vars['amount']
|
||||
self._currency = template_vars['currency']
|
||||
if self._currency in self.signed_currencies:
|
||||
self._amt_fmt = self.signed_currency_fmt
|
||||
else:
|
||||
self._amt_fmt = self.unsigned_currency_fmt
|
||||
if percentage is None:
|
||||
amount = self._remainder
|
||||
else:
|
||||
amount = decimal.Decimal(babel.numbers.format_currency(
|
||||
template_vars['amount'] * percentage, self._currency, '###0.###'))
|
||||
self._remainder -= amount
|
||||
amt_s = babel.numbers.format_currency(amount, self._currency, self._amt_fmt)
|
||||
return ' {:45} {:>19}\n'.format(account, amt_s)
|
||||
if template_vars is not self._last_template_vars:
|
||||
self._split_iter = self._iter_splits(template_vars)
|
||||
self._last_template_vars = template_vars
|
||||
return next(self._split_iter)
|
||||
|
||||
|
||||
class Template:
|
||||
ACCOUNT_SPLIT_RE = re.compile(r'(?:\t| )\s*')
|
||||
DATE_FMT = '%Y/%m/%d'
|
||||
SIGNED_CURRENCY_FMT = '¤#,##0.###;¤-#,##0.###'
|
||||
UNSIGNED_CURRENCY_FMT = '#,##0.### ¤¤'
|
||||
|
|
@ -51,12 +155,11 @@ class Template:
|
|||
def __init__(self, template_s, signed_currencies=frozenset(),
|
||||
date_fmt=DATE_FMT,
|
||||
signed_currency_fmt=SIGNED_CURRENCY_FMT,
|
||||
unsigned_currency_fmt=UNSIGNED_CURRENCY_FMT):
|
||||
unsigned_currency_fmt=UNSIGNED_CURRENCY_FMT,
|
||||
template_name='<template>'):
|
||||
self.date_fmt = date_fmt
|
||||
self.pos_splits = AccountSplitter(
|
||||
1, signed_currencies, signed_currency_fmt, unsigned_currency_fmt)
|
||||
self.neg_splits = AccountSplitter(
|
||||
-1, signed_currencies, signed_currency_fmt, unsigned_currency_fmt)
|
||||
self.splitter = AccountSplitter(
|
||||
signed_currencies, signed_currency_fmt, unsigned_currency_fmt, template_name)
|
||||
|
||||
lines = [s.lstrip() for s in template_s.splitlines(True)]
|
||||
for index, line in enumerate(lines):
|
||||
|
|
@ -72,10 +175,14 @@ class Template:
|
|||
if line[0].isalpha():
|
||||
if start_index < index:
|
||||
self._add_str_func(lines[start_index:index])
|
||||
account, percentage_s = line.rsplit(None, 1)
|
||||
splitter = self.neg_splits if percentage_s.startswith('-') else self.pos_splits
|
||||
splitter.add(account, percentage_s)
|
||||
self.format_funcs.append(splitter.render_next)
|
||||
line = line.strip()
|
||||
match = self.ACCOUNT_SPLIT_RE.search(line)
|
||||
if match is None:
|
||||
raise errors.UserInputError("no amount expression found", line)
|
||||
account = line[:match.start()]
|
||||
amount_expr = line[match.end():]
|
||||
self.splitter.add(account, amount_expr)
|
||||
self.format_funcs.append(self.splitter.render_next)
|
||||
start_index = index + 1
|
||||
if start_index <= index:
|
||||
self._add_str_func(lines[start_index:])
|
||||
|
|
|
|||
|
|
@ -1,22 +1,29 @@
|
|||
[Simplest]
|
||||
template =
|
||||
Accrued:Accounts Receivable 100%%
|
||||
Income:Donations -100%%
|
||||
Accrued:Accounts Receivable {amount}
|
||||
Income:Donations -{amount}
|
||||
|
||||
[FiftyFifty]
|
||||
template =
|
||||
Accrued:Accounts Receivable 100%%
|
||||
Income:Donations -50%%
|
||||
Income:Sales -50%%
|
||||
Accrued:Accounts Receivable {amount}
|
||||
Income:Donations -.5 * {amount}
|
||||
Income:Sales -.5*{amount}
|
||||
|
||||
[Complex]
|
||||
template =
|
||||
;Tag: Value
|
||||
;TransactionID: {txid}
|
||||
Accrued:Accounts Receivable 100%%
|
||||
Accrued:Accounts Receivable {amount}
|
||||
;Entity: Supplier
|
||||
Income:Donations:Specific -95.5%%
|
||||
Income:Donations:Specific -.955* {amount}
|
||||
;Program: Specific
|
||||
;Entity: {entity}
|
||||
Income:Donations:General -4.5%%
|
||||
Income:Donations:General -.045 * {amount}
|
||||
;Entity: {entity}
|
||||
|
||||
[Multivalue]
|
||||
template =
|
||||
Expenses:Taxes {tax}
|
||||
Accrued:Accounts Receivable {amount} - {tax}
|
||||
Income:RBI -.1*{amount}
|
||||
Income:Donations -.9*{amount}
|
||||
|
|
|
|||
|
|
@ -5,15 +5,15 @@ signed_currencies = EUR
|
|||
[Templates]
|
||||
output_path = Template.output
|
||||
one =
|
||||
Accrued:Accounts Receivable 100%%
|
||||
Income:Donations -100%%
|
||||
Accrued:Accounts Receivable {amount}
|
||||
Income:Donations -{amount}
|
||||
two =
|
||||
;Tag1: {value}
|
||||
Income:Donations -100%%
|
||||
Income:Donations -{amount}
|
||||
;IncomeTag: Donations
|
||||
Expenses:Fundraising 5%%
|
||||
Expenses:Fundraising .05 * {amount}
|
||||
;ExpenseTag: Fundraising
|
||||
Accrued:Accounts Receivable 95%%
|
||||
Accrued:Accounts Receivable .95 * {amount}
|
||||
;AccrualTag: Receivable
|
||||
|
||||
[Date]
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ signed_currencies = USD
|
|||
|
||||
[One]
|
||||
template patreon cardfees =
|
||||
Accrued:Accounts Receivable -100%%
|
||||
Expenses:Fees:Credit Card 100%%
|
||||
Accrued:Accounts Receivable -{amount}
|
||||
Expenses:Fees:Credit Card {amount}
|
||||
template patreon svcfees =
|
||||
Accrued:Accounts Receivable -100%%
|
||||
Expenses:Fundraising 100%%
|
||||
Accrued:Accounts Receivable -{amount}
|
||||
Expenses:Fundraising {amount}
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ def test_defaults():
|
|||
assert kwargs == {
|
||||
'date_fmt': '%Y-%m-%d',
|
||||
'signed_currency_fmt': kwargs['signed_currency_fmt'],
|
||||
'template_name': 'one',
|
||||
'unsigned_currency_fmt': kwargs['unsigned_currency_fmt'],
|
||||
}
|
||||
|
||||
|
|
@ -43,7 +44,7 @@ def test_template_parsing():
|
|||
except IndexError as error:
|
||||
assert False, error
|
||||
assert "\n;Tag1: {value}\n" in tmpl_s
|
||||
assert "\nIncome:Donations -100%\n" in tmpl_s
|
||||
assert "\nIncome:Donations -{amount}\n" in tmpl_s
|
||||
assert "\n;IncomeTag: Donations\n" in tmpl_s
|
||||
|
||||
@pytest.mark.parametrize('arg_s', [None, '-', 'output.ledger'])
|
||||
|
|
|
|||
|
|
@ -72,3 +72,17 @@ def test_balancing():
|
|||
" Income:Donations -0.50 USD",
|
||||
" Income:Sales -0.51 USD",
|
||||
]
|
||||
|
||||
def test_multivalue():
|
||||
tmpl = template_from('Multivalue')
|
||||
rendered = tmpl.render('DD', decimal.Decimal('150.00'), 'USD', DATE,
|
||||
tax=decimal.Decimal('12.50'))
|
||||
lines = [normalize_whitespace(s) for s in rendered.splitlines()]
|
||||
assert lines == [
|
||||
"",
|
||||
"2015/03/14 DD",
|
||||
" Expenses:Taxes 12.50 USD",
|
||||
" Accrued:Accounts Receivable 137.50 USD",
|
||||
" Income:RBI -15.00 USD",
|
||||
" Income:Donations -135.00 USD",
|
||||
]
|
||||
|
|
|
|||
Loading…
Reference in a new issue