diff --git a/CODE.rst b/CODE.rst index abeafb0..2f092d5 100644 --- a/CODE.rst +++ b/CODE.rst @@ -39,9 +39,6 @@ Class method ``can_handle(source_file)`` ``__iter__()`` Returns a iterator of entry data dicts. -Class attribute ``TEMPLATE_KEY`` - A string with the full key to load the corresponding template from the user's configuration (e.g., ``'template patreon income'``). - Hooks ~~~~~ @@ -62,17 +59,6 @@ Hooks make arbitrary transformations to entry data dicts. Every entry data dict Class attribute ``KIND`` This should be one of the values of the ``hooks.HOOK_KINDS`` enum. This information determines what order hooks run in. -Templates -~~~~~~~~~ - -Templates receive entry data dicts and format them into final output entries. - -``__init__(template_str)`` - Initializes the template from a single string, as read from the user's configuration. - -``render(entry_data)`` - Returns a string with the output entry, using the given entry data. - Loading importers and hooks --------------------------- diff --git a/README.rst b/README.rst index b9a0c46..1cdf6f5 100644 --- a/README.rst +++ b/README.rst @@ -29,7 +29,7 @@ A template looks like a Ledger entry with a couple of differences: Here's a simple template for Patreon patron payments:: [DEFAULT] - template patreon income = + patreon income ledger entry = ;Tag: Value Income:Patreon -{amount} Accrued:Accounts Receivable:Patreon {amount} @@ -38,7 +38,7 @@ Let's walk through this line by line. Every setting in your configuration file has to be in a section. ``[DEFAULT]`` is the default section, and import2ledger reads configuration settings from here if you don't specify another one. This documentation explains how to use sections later. -``template patreon income =`` specifies which entry template this is. Every template is found from a setting with a name in the pattern ``template ``. The remaining lines are indented further than this name; this defines a multiline value. Don't worry about the exact indentation of your template; import2ledger will indent its output nicely. +``patreon income ledger entry =`` specifies which entry template this is. Every template is found from a setting with a name in the pattern `` ledger entry``. The remaining lines are indented further than this name; this defines a multiline value. Don't worry about the exact indentation of your template; import2ledger will indent its output nicely. The first line of the template is a Ledger tag. The program will leave all kinds of tags and Ledger comments alone, except to indent them nicely. @@ -104,13 +104,13 @@ You can define the following templates. Amazon Affiliate Program ^^^^^^^^^^^^^^^^^^^^^^^^ -``template amazon earnings`` +``amazon earnings ledger entry`` Imports one transaction per month, summarizing all earnings over the month. Generated from Amazon's "Fee Earnings" report CSV. Benevity ^^^^^^^^ -``template benevity payments`` +``benevity donation ledger entry`` Imports one transaction per row in Benevity's donations report CSV. This template can use these variables: @@ -140,16 +140,16 @@ Benevity Patreon ^^^^^^^ -``template patreon income`` +``patreon income ledger entry`` Imports one transaction per patron per month. Generated from Patreon's monthly patron report CSVs. -``template patreon cardfees`` +``patreon cardfees ledger entry`` Imports one expense transaction per month for that month's credit card fees. Generated from Patreon's earnings report CSV. -``template patreon svcfees`` +``patreon servicefees ledger entry`` Imports one expense transaction per month for that month's Patreon service fees. Generated from Patreon's earnings report CSV. -``template patreon vat`` +``patreon vat ledger entry`` Imports one transaction per country per month each time Patreon withheld VAT. Generated from Patreon's VAT report CSV. This template can use these variables: @@ -166,7 +166,7 @@ Patreon Stripe ^^^^^^ -``template stripe payments`` +``stripe payment ledger entry`` Imports one transaction per payment. Generated from Stripe's payments CSV export. This template can use these variables: diff --git a/TODO.rst b/TODO.rst index 0185219..8b242c9 100644 --- a/TODO.rst +++ b/TODO.rst @@ -4,19 +4,10 @@ TODO Template multiplexing with action hooks --------------------------------------- -The big idea: make it easier for hooks to customize *what* template(s) are rendered by moving more of the process into hooks—including template rendering itself. - -Required: - -* Make the main loop seed the entry data with information about the importer used. - -* Move template rendering into a hook, where the template to load is determined by a value in the entry data. - -Extra customizations after that's done: - * Add a hook that simply reads information from a configuration file section ``[template variables]`` and adds it to the entry data. * Add a hook that changes what template to use based on other entry data. (This needs more specification.) + OR, should this be a hook that just adds more entry data, and trusts the Ledger entry template to use it? New importers ------------- diff --git a/import2ledger/__main__.py b/import2ledger/__main__.py index 57ec1e9..5d7896d 100644 --- a/import2ledger/__main__.py +++ b/import2ledger/__main__.py @@ -34,8 +34,9 @@ class FileImporter: 'source_stem': in_path.stem, } for importer in importers: + source_vars['importer_class'] = importer.__name__ + source_vars['importer_module'] = importer.__module__ in_file.seek(0) - source_vars['template'] = importer.TEMPLATE_KEY for entry_data in importer(in_file): entry_data = collections.ChainMap(entry_data, source_vars) for hook in self.hooks: diff --git a/import2ledger/hooks/ledger_entry.py b/import2ledger/hooks/ledger_entry.py index 46b46b2..97584d9 100644 --- a/import2ledger/hooks/ledger_entry.py +++ b/import2ledger/hooks/ledger_entry.py @@ -283,7 +283,7 @@ class LedgerEntryHook: template_s = section_config[config_key] except KeyError: raise errors.UserInputConfigurationError( - "template not defined in [{}]".format(section_name), + "Ledger template not defined in [{}]".format(section_name), config_key, ) return Template( @@ -297,9 +297,16 @@ class LedgerEntryHook: def run(self, entry_data): try: - template = self._load_template(self.config, None, entry_data['template']) + template_key = entry_data['ledger template'] + except KeyError: + template_key = '{} {} ledger entry'.format( + strparse.rslice_words(entry_data['importer_module'], -1, '.', 1), + entry_data['importer_class'][:-8].lower(), + ) + try: + template = self._load_template(self.config, None, template_key) except errors.UserInputConfigurationError as error: - if error.strerror.startswith('template not defined '): + if error.strerror.startswith('Ledger template not defined '): have_template = False else: raise diff --git a/import2ledger/importers/amazon.py b/import2ledger/importers/amazon.py index 6927e97..fc8202e 100644 --- a/import2ledger/importers/amazon.py +++ b/import2ledger/importers/amazon.py @@ -43,7 +43,6 @@ class EarningsImporter(_csv.CSVImporterBase): 'Date Shipped': '1979-07-09', } NEEDED_FIELDS = frozenset(SENTINEL_ROW.keys()) - TEMPLATE_KEY = 'template amazon earnings' ENTRY_SEED = { 'currency': 'USD', 'payee': 'Amazon', diff --git a/import2ledger/importers/benevity.py b/import2ledger/importers/benevity.py index c304103..8b29ebd 100644 --- a/import2ledger/importers/benevity.py +++ b/import2ledger/importers/benevity.py @@ -1,7 +1,7 @@ from . import _csv from .. import strparse -class PaymentImporter(_csv.CSVImporterBase): +class DonationsImporter(_csv.CSVImporterBase): HEADER_FIELDS = { 'Currency': 'currency', 'Disbursement ID': 'disbursement_id', @@ -26,7 +26,6 @@ class PaymentImporter(_csv.CSVImporterBase): 'Transaction ID': 'transaction_id', 'Donation Frequency': 'frequency', } - TEMPLATE_KEY = 'template benevity payments' DATE_FMT = '%Y-%m-%d' NOT_SHARED = 'Not shared by donor' diff --git a/import2ledger/importers/nbpy2017.py b/import2ledger/importers/nbpy2017.py index d871fae..b99a7da 100644 --- a/import2ledger/importers/nbpy2017.py +++ b/import2ledger/importers/nbpy2017.py @@ -121,7 +121,10 @@ def _parse_invoice(parser_class, source_file): except AttributeError: return None -class ImporterBase: +class InvoiceImporter: + INVOICE_CLASS = Invoice2017 + LEDGER_TEMPLATE_KEY_FMT = 'nbpy2017 {0} ledger entry' + @classmethod def _parse_invoice(cls, source_file): return _parse_invoice(cls.INVOICE_CLASS, source_file) @@ -135,17 +138,5 @@ class ImporterBase: def __iter__(self): for entry in self.invoice: - if entry['status'] == self.YIELD_STATUS: - yield entry - - -class Invoice2017Importer(ImporterBase): - TEMPLATE_KEY = 'template nbpy2017 invoice' - INVOICE_CLASS = Invoice2017 - YIELD_STATUS = STATUS_INVOICED - - -class Payment2017Importer(ImporterBase): - TEMPLATE_KEY = 'template nbpy2017 payment' - INVOICE_CLASS = Invoice2017 - YIELD_STATUS = STATUS_PAID + entry['ledger entry'] = self.LEDGER_TEMPLATE_KEY_FMT.format(entry['status'].lower()) + yield entry diff --git a/import2ledger/importers/patreon.py b/import2ledger/importers/patreon.py index 1df117f..ae347be 100644 --- a/import2ledger/importers/patreon.py +++ b/import2ledger/importers/patreon.py @@ -17,7 +17,6 @@ class IncomeImporter(_csv.CSVImporterBase): ENTRY_SEED = { 'currency': 'USD', } - TEMPLATE_KEY = 'template patreon income' def __init__(self, input_file): super().__init__(input_file) @@ -48,16 +47,14 @@ class FeeImporterBase(_csv.CSVImporterBase): } -class PatreonFeeImporter(FeeImporterBase): +class ServiceFeesImporter(FeeImporterBase): AMOUNT_FIELD = 'Patreon Fee' NEEDED_FIELDS = frozenset(['Month', AMOUNT_FIELD]) - TEMPLATE_KEY = 'template patreon svcfees' -class CardFeeImporter(FeeImporterBase): +class CardFeesImporter(FeeImporterBase): AMOUNT_FIELD = 'Processing Fees' NEEDED_FIELDS = frozenset(['Month', AMOUNT_FIELD]) - TEMPLATE_KEY = 'template patreon cardfees' class VATImporter(FeeImporterBase): @@ -67,4 +64,3 @@ class VATImporter(FeeImporterBase): 'Country Code': 'country_code', 'Country Name': 'country_name', } - TEMPLATE_KEY = 'template patreon vat' diff --git a/import2ledger/importers/stripe.py b/import2ledger/importers/stripe.py index 99b03c4..1c91e26 100644 --- a/import2ledger/importers/stripe.py +++ b/import2ledger/importers/stripe.py @@ -17,7 +17,6 @@ class PaymentImporter(_csv.CSVImporterBase): 'Description': 'description', 'id': 'payment_id', } - TEMPLATE_KEY = 'template stripe payments' DATE_FMT = '%Y-%m-%d' def _read_row(self, row): diff --git a/tests/data/imports.yml b/tests/data/imports.yml index 1b7f967..420a719 100644 --- a/tests/data/imports.yml +++ b/tests/data/imports.yml @@ -11,7 +11,7 @@ currency: USD - source: PatreonEarnings.csv - importer: patreon.PatreonFeeImporter + importer: patreon.ServiceFeesImporter expect: - payee: Patreon date: !!python/object/apply:datetime.date [2017, 9, 1] @@ -23,7 +23,7 @@ currency: USD - source: PatreonEarnings.csv - importer: patreon.CardFeeImporter + importer: patreon.CardFeesImporter expect: - payee: Patreon date: !!python/object/apply:datetime.date [2017, 9, 1] @@ -83,9 +83,10 @@ description: "Payment for invoice #100" - source: nbpy2017a.html - importer: nbpy2017.Invoice2017Importer + importer: nbpy2017.InvoiceImporter expect: - payee: Python Person A + ledger entry: nbpy2017 invoice ledger entry date: !!python/object/apply:datetime.date [2017, 10, 19] amount: !!python/object/apply:decimal.Decimal ["80.00"] tickets_sold: !!python/object/apply:decimal.Decimal ["1"] @@ -96,41 +97,8 @@ status: Invoice invoice_id: "83" invoice_date: !!python/object/apply:datetime.date [2017, 10, 19] - -- source: nbpy2017b.html - importer: nbpy2017.Invoice2017Importer - expect: - - payee: Python Person B - date: !!python/object/apply:datetime.date [2017, 12, 3] - amount: !!python/object/apply:decimal.Decimal ["50.00"] - tickets_sold: !!python/object/apply:decimal.Decimal ["1"] - ticket_rate: !!python/object/apply:decimal.Decimal ["42.50"] - shirts_sold: !!python/object/apply:decimal.Decimal ["0"] - shirt_rate: !!python/object/apply:decimal.Decimal ["25.50"] - status: Invoice - currency: USD - invoice_date: !!python/object/apply:datetime.date [2017, 12, 3] - invoice_id: "304" - -- source: nbpy2017c.html - importer: nbpy2017.Invoice2017Importer - expect: - - payee: Python Person C - date: !!python/object/apply:datetime.date [2017, 10, 5] - amount: !!python/object/apply:decimal.Decimal ["55.00"] - tickets_sold: !!python/object/apply:decimal.Decimal ["1"] - ticket_rate: !!python/object/apply:decimal.Decimal ["21.25"] - shirts_sold: !!python/object/apply:decimal.Decimal ["1"] - shirt_rate: !!python/object/apply:decimal.Decimal ["25.50"] - status: Invoice - currency: USD - invoice_date: !!python/object/apply:datetime.date [2017, 10, 5] - invoice_id: "11" - -- source: nbpy2017a.html - importer: nbpy2017.Payment2017Importer - expect: - payee: Python Person A + ledger entry: nbpy2017 payment ledger entry date: !!python/object/apply:datetime.date [2017, 10, 19] amount: !!python/object/apply:decimal.Decimal ["80.00"] tickets_sold: !!python/object/apply:decimal.Decimal ["1"] @@ -145,9 +113,22 @@ stripe_id: ch_ahr0ue8lai1ohqu4Gei4Biem - source: nbpy2017b.html - importer: nbpy2017.Payment2017Importer + importer: nbpy2017.InvoiceImporter expect: - payee: Python Person B + ledger entry: nbpy2017 invoice ledger entry + date: !!python/object/apply:datetime.date [2017, 12, 3] + amount: !!python/object/apply:decimal.Decimal ["50.00"] + tickets_sold: !!python/object/apply:decimal.Decimal ["1"] + ticket_rate: !!python/object/apply:decimal.Decimal ["42.50"] + shirts_sold: !!python/object/apply:decimal.Decimal ["0"] + shirt_rate: !!python/object/apply:decimal.Decimal ["25.50"] + status: Invoice + currency: USD + invoice_date: !!python/object/apply:datetime.date [2017, 12, 3] + invoice_id: "304" + - payee: Python Person B + ledger entry: nbpy2017 payment ledger entry date: !!python/object/apply:datetime.date [2017, 12, 3] amount: !!python/object/apply:decimal.Decimal ["50.00"] tickets_sold: !!python/object/apply:decimal.Decimal ["1"] @@ -162,9 +143,22 @@ invoice_id: "304" - source: nbpy2017c.html - importer: nbpy2017.Payment2017Importer + importer: nbpy2017.InvoiceImporter expect: - payee: Python Person C + ledger entry: nbpy2017 invoice ledger entry + date: !!python/object/apply:datetime.date [2017, 10, 5] + amount: !!python/object/apply:decimal.Decimal ["55.00"] + tickets_sold: !!python/object/apply:decimal.Decimal ["1"] + ticket_rate: !!python/object/apply:decimal.Decimal ["21.25"] + shirts_sold: !!python/object/apply:decimal.Decimal ["1"] + shirt_rate: !!python/object/apply:decimal.Decimal ["25.50"] + status: Invoice + currency: USD + invoice_date: !!python/object/apply:datetime.date [2017, 10, 5] + invoice_id: "11" + - payee: Python Person C + ledger entry: nbpy2017 payment ledger entry date: !!python/object/apply:datetime.date [2017, 10, 5] amount: !!python/object/apply:decimal.Decimal ["55.00"] tickets_sold: !!python/object/apply:decimal.Decimal ["1"] @@ -191,7 +185,7 @@ currency: USD - source: Benevity.csv - importer: benevity.PaymentImporter + importer: benevity.DonationsImporter expect: - date: !!python/object/apply:datetime.date [2017, 10, 28] currency: USD diff --git a/tests/data/test_main.ini b/tests/data/test_main.ini index 5babf5c..1d4dfb2 100644 --- a/tests/data/test_main.ini +++ b/tests/data/test_main.ini @@ -4,10 +4,10 @@ loglevel = critical signed_currencies = USD [One] -template patreon cardfees = +patreon cardfees ledger entry = Accrued:Accounts Receivable -{amount} Expenses:Fees:Credit Card {amount} -template patreon svcfees = +patreon servicefees ledger entry = ;SourcePath: {source_abspath} ;SourceName: {source_name} Accrued:Accounts Receivable -{amount} diff --git a/tests/test_hook_ledger_entry.py b/tests/test_hook_ledger_entry.py index e748be0..feb876c 100644 --- a/tests/test_hook_ledger_entry.py +++ b/tests/test_hook_ledger_entry.py @@ -27,7 +27,7 @@ def template_vars(payee, amount, currency='USD', date=DATE, other_vars=None): 'currency': currency, 'date': date, 'payee': payee, - 'template': 'template', + 'ledger template': 'template', } if other_vars is None: return call_vars