reports should include an option to output the manager version used to create that report
reports should be able to be “flagged” and a snapshot created of the values in that report and those values stored separately within the database.
This is the key feature request though:
upon manager and database updates (engine upgrades and flawed behaviour rectifications), those flagged reports should be re-run and discrepancies highlighted to the user so they can be rectified
I wish I had kept printed copies of reports I’d generated, even saving off the PDF and attaching back to manager would suffice, but I keep thinking “the data is not going to change”…
But it does. And those reports are lost forever.
Yes, you can easily blame the user for not using manager correctly or relying on flawed behaviour, but at the end of the day, you need to satisfy us, auditors, creditors, shareholders, prospective business partners, or buyers, (oh, and tax authorities) is still correct.
Reports exist in Manager only as a set of defined parameters, such as the date range. The data generated, organized, processed and displayed by the definition is not stored anywhere. The report is recreated every time you view it. So what you suggest is not feasible.
Which is why you should comment on the accounting side and not the development side.
What you said prior to that sentence is entirely correct and how manager functions. What i am suggesting is merely an extension of something that is already done, let me explain:
As you rightly stated, every report is merely a collection of queries and a resultant data set, from that dataset a report is generated.
The reports are little more than a stored query, whether they are stored in the database or the application, it’s of little concern, what is important is that there is a record/query and it’s reproducible.
I am suggesting that the resultant dataset become a new table/record in the database. This serves several purposes:
It creates a snapshot, a static store of that report, and
It keeps the integrity of that data,
It also enforces updates between version be honest about changes whereby when an underlying fundamental feature does change, it will notify the user.
To that last point, no upgrade of any version should change any underlying output and database structure without first making this report to the user.
Then when @luboschanges or fixes some underlying bad behaviour, those “snapshot” reports can be run again and the database is validated, or reconciled (so to speak) and the update progresses and changes can continue. When the result set does change, you know where, when and why. You can report that to those concerned and everything is koscher.
When they can’t be run again (because they fundamentally changed, such as the report now relies on new columns not previously within the query, or something becomes inclusive instead of exclusive, or it’s a completely new query, etc, ie, for ANY reason), the user is presented with the new vs old query and the user can make decisions with informed consent. You know when the data breaks. None of this, where’s my data gone or why isn’t this calculated anymore.
When it’s only the resultant dataset that’s changed, these can be flagged to the user and the problems can be identified. “Your sales Are up for that period? That can’t be, oh, now the tax is calculated this way instead of that, that’s not right,” and the user either:
updates and records the data correctly (if it was a user error),
reports it to lubos (if it was a development error), or
has a record of that discrepancy and can report the relevant changes as required to the appropriate authorities (this is the rare case that the data is recorded correctly and manager previously output incorrect data, no extra data manipulation is required, but journal entries may need to be made or “whatever”)
I’m not talking about storing every data that manager generates, that would be recursively endless, but what I am talking about is fundamental reports such as P&L, BS, and BAS/Tax reports at specified periods. A finite/limited set. Like bank reconciliations.
Having come from a digital forensics background where data integrity is paramount for presentation at court, we performed numerous “hashes” not only of the forensic image, but validation of the tools themselves. If, for example, MD5 hashes were Replaced with SHA256 hashes and no backward compatibility / reference table existed, that forensically would be negligent, even reckless and would open the analyst to prosecution. Analysts wouldn’t make such a change because the forensic image would become unusable. Invalidated, and the court would not accept the evidence.
A tool (manager) that presents “perpetual” legal documents needs these checks and balances.
Patches suggestion (and I have previously suggested exactly the same model) is broken because it relies on the user physically taking those printed (displayed) documents and physically comparing new for old at every update) a hugely error prone task. And this would be daily in the case of cloud users! This task should be automated.
It is also broken because managers paradigm is past versions should break and this is reproducible by starting any old version of manager in its current location, it fails with a fatal error because the database has been corrupted with new data. (Actually it hasn’t, but a hidden data file stores database versioning and location information in a hidden path. If lubos didn’t obfuscate this and used a proper configuration file where users could set multiple versions and independent database locations based on those versions, then multiple versions of manager could be run simultaneously and those different databases compared. However this is decidedly and intentionally not supported. An impossible task for the average user. Think about this: if you take a backup, then update your application, then open the database, don’t like what you see, you can’t simply close the database, close manager and go back to the old version. The old version of manager is “invalidated” and fails fatally by design. Users can work around this by deleting the hidden database versioning information and deleting the old database, then reopen the old file and re-import the data file. Convoluted and IMO, stupid. What should happen, create your backup and set a new data location for the new versions database. Close manager. Update and reopen manager and reimport the old database into the new manager or upgrade the existing one in the new location. Then, if anything goes wrong, users can “reset” to the old version and old location and open everything unfettered). Any user should be able to open two versions of manager at the same time with different databases in different locations to compare the results. Without that ability and without data integrity, manager is now little more than a toy. Without perpetual data integrity, it shouldn’t purport to be perpetual. It should be closed, locked off and a new database created for each year.
Although that’s not to say that lubos couldn’t develop a tool that compares different backup versions and performs a diff over them, but that’s not Anything like what I’m suggesting.
I’m suggesting transparency beyond “it’s hidden in the release notes behind “experimental engine” BS”
I work on the principle that if program is bug-free, the figures won’t change. This feature would be pointless in the long-term because figures won’t change.
Sorry lubos, but your principles are flawed because the figures do change and because you will never be bug free. It’s just not possible. balanced invoices is just one of my examples but there are dozens (probably hundreds) littered around the forum.
I say this with empathy and compassion. I do understand the difficulties but you are deluded if you think that the application is bug free and numbers won’t change.
Every bug is put into bugs category. There are currently 6 topics. 4 are related to internationalization, 1 is crashing issue on Linux and one is missing feature. None of them when fixed will affect the figures.
I don’t know about dozens of bugs littered around the forum. Fixing bugs has been always my top priority.
We are arguing semantics here, I just said you’ll never be free of bugs but I was specifically talking about were “dozens of examples”, not “official bugs”.
We are coming from different ends of the argument. What I am calling a bug (I prefer “change” or “presentation issues”) you call a myriad of terms: user error, engine upgrades, flawed or unintended behaviour, etc. You put the onus back on the user to not use and adapt your program contrary to your design. You are not taking into consideration how it is used in the broader community, especially when it comes to features you don’t, won’t or haven’t yet implemented.
We create work arounds for those lacking/missing features, and you then break and/or change the presentation of that data based on how you intended to interpret the data and not how the user actually presented and intended it to be used in their business model. That (to us) is flawed.
When I say “bug”, “change”, “broken" or “example”, I am referring to any and all of (and not limited to):
any reports you’ve deleted or reformatted
any change to implementation or calculation of taxes
any underlying assumptions you have made about user data and how it should be presented/interpreted
any underlying queries you’ve modified
user generated custom reports you’ve deprecated,
worksheets you’ve updated
the bad programming paradigms and behaviours you’ve fixed…
engines you’ve “upgraded”, or replaced, and yes,
bugs you have fixed, as well as
current unfixed bugs
Whatever you call it, it is a frustration we don’t need and you need to take some responsibility for the representation, and changes to the representation of our data.
Users have wasted hours and days creating custom reports that you deprecated in one fell swoop.
What happens when you implement the payments and receipts tax changes you’ve posed for years? What users work arounds are going to fall apart and break existing and previously submitted reports?
You may not be changing the actual data we enter (if we enter 5 units, you record 5 units), but you are changing its implementation and presentation which we rely on for legal purposes. I’m very surprised you are missing that very important point.
It is not just the raw data and its storage but also the accompanying implementation and presentation.
You state:
But they do, and they have and so this feature is anything but pointless.
You say the figures will always be right and won’t change, but they will only “not change” if you do not change the underlying structure, and if you do, you need a way to identify that change. That is exactly my point.
In my other thread you just commented on exactly why you would change the output (and rightly so):
And that is my point EXACTLY!
That’s what I’m asking for, a mechanism that reports when those differences appear. Because bugs do get found. You can’t presume you are immune.
That was precisely the point of this whole thread. It is exactly why I’m frustrated with manager.
My previous example was when my sales data balances and tax reporting fails because you didn’t cater for balances invoices, that is “an example”. Yes it’s now fixed (and then it wasn’t, and now it is again), but it makes all my previous BAS submissions wrong. In no way can you consider those previous presentations “right”. And on this last update (again, right as I go to see my accountant) you changed the implementation of tax calculations again which changed my data (in this case, the change was in error, but that takes me days to get to the bottom of. However, unless I run those reports again at a later time, AND MANUALLY COMPARE THEM TO PREVIOUSLY SUBMITTED REPORTS, I’m not going to readily pick that up until next tax time. That’s why I would like a snapshot of important reports. So that I can report on the errors as they are found.
BTW I am happy to have this conversation elsewhere
Are you currently doing any workaround for currently lacking/missing feature? If so, what is it? I need to know specifics.
Manager is being used by accountants and by non-accountants. It’s non-accountants who are being very “creative” with Manager and as a result - fall into traps. I need to calibrate the system so it nudges (even the most creative non-accountants) in the right direction so they end up doing roughly the same thing an accountant would do.
Assuming the software is bug-free and you do the right thing, your figures won’t change.
I’m going to call you out on this. How did this specific bug make all your BAS submissions wrong? You have discovered a bug with zero-valued invoices on cash-basis and it was fixed within 6 hours.
@d3mad I appreciate you are frustrated and have lost a significant amount of your time to upgrade your Manager data file and maintain compatibility with the current version of Manager.
I maybe grossly misinterpreting your suggestion and subsequent comments however the feature request you are suggesting comes across as:
No Manager program improvement can be made if program changes may alter the effect of any archived entry in my Manager data file irrespective of how creative I have been with data entry or if the old calculation is wrong. Manager can not introduce and enhanced user experience if some user intervention maybe required.
While it is theoretically possible to implement that type of update policy, doing so would be a huge impost on software development. The recent update of production costing probably would not have been viable to implement. It is also boarding on insulting to the program designer. My understanding is changes are carefully considered prior to implementation (the reasons for some only being apparent to users like me after future changes a released). I do not believe Lubos makes significant program changes on a whim.
Having said that I can see merit in a generalization of your suggestion, a side effect of which is it would also help detect any program produced changes in Managers output.
Idea (which maybe just a rewording if your suggestion)
What if every time Manager emailed, printed, or created a pdf then metadata for the output was saved to the “History” (as distinct to document creation parameters). Then when a user was in that document “View” window, history would be available, and displaying the history would highlight any changes from the current dynamic document.
This should apply to at least invoices, payslips, and tax reports.
I do not know how hard it would be to implement but it would be generally useful in my opinion. Mostly for user created changes and ensuring I had an exact record of any document I have sent out, but it would also catch any program created changes.
@lubos, I have not yet updated Manager to a version that is affected by this change in sales and purchase tax calculation so currently am not affected. However I have been following the threads with great interest and would like to add my input because I agree with both sides of the argument.
Your example of Manager calculating the VAT return incorrectly in previous years and corrected today makes you see that you suddenly owe the taxman more money that you had thought. I agree with you. The taxman will want the missing money. No disagreement here.
However, lets say I am on the latest version of Manager today and it changes my VAT, corporation tax, starting balances etc etc for previous years. Now the taxman wants to audit my company.
He will look and see that I have filed x amounts for VAT returns for each quarter over the last five years and then looks at Manager and sees that the amounts are different. Some thing with corporation tax, profit and loss figures for end of year etc. So he will ask me why I am cooking the books as my accounts in Manager no longer match up with what I filed. You say that the program has simply fixed bugs - which I understand and have no problem with. But the auditor will not know whether I am cooking my books or whether the program has changed things.
So I fully agree with you that Manager should fix bugs in the past and correctly show what tax you now owe to the taxman. But I also agree with users that you can’t simply change previous years submissions because in the event of an audit Manager reports no longer reconcile with what I have submitted to the taxman in previous years. How do they know whether the program fixed something or whether I have committed fraud by fiddling the figures?
I am not in favour of your suggestion of having multiple versions of Manager open (which you mentioned as this would complicate things for non accountants and people who are not good with computers and may end up with people updating the wrong version of Manager. I really can’t see the point in this suggestion.
But I am in agreement with you that because Manager is a perpetual system, not an accounting program where you formally close one year end and start a new accounting period that it would be impossible to have Manager designed so that in this year Manager reports calculates the figures using the new algorithm and in previous years, it calculates using the old algorithm. I know enough about programming to know that this is simply impractical as the program will continue to evolve over years, fixing bugs etc and it would be a minefield.
So I agree with your suggestion that Manager should fix the bug and enable you to pay/receive the difference in tax to/from the taxman. However, there still needs to be some way for users to preserve previous financial year’s submissions that they have made to the taxman.
What I would propose for VAT returns for example is when you create the VAT return and “submit” it, that it should create a pdf attachment linked to that report to create a snapshot of that submission at that point in time. The same thing could be done with profit and loss, balance sheet reports at year end - to create a “snapshot” of the report in pdf format that will remain unchanged regardless of future changes. Once you have these pdf Snapshots, then the next step will be to work out how to get Manager to make users aware that their starting balances, Vat return values etc have changed after an update, so that users could make a journal entry adjustment in previous years to allow for the changes? I am in agreement with you that a journal entry in previous years to “balance” things out so that it matches what it used to be is necessary. But in order to create a journal entry, you a: need to be aware that your figures have changed and b: you need to have a snapshot of what the figures were before the Manager update.
I do not see how you can expect the taxman to simply believe that the reason your accounts no longer reconcile with what you filed as your VAT and end of year return is as result of fixing bugs in the accounting program. If I am auditing you and the figures no longer reconcile, I would assume that you are cooking your books.
I agree with your point of view, but users are also correct in stating that Manager needs to preserve snapshots of previous years submissions so as not to be seen to be cooking the books. I am hoping that my suggestion of creating pdf snapshots and finding some way for Manager to alert users that their starting balances, Vat returns etc have changed after an update will address the concerns of all parties. I am also very concerned about a situation where I filed tax returns, manager then changes the calculations and an audit would say - these figures do not reconcile with what is filed.
I would also add (which others have raised over the years), please implement a small feedback panel comprised of end users who can provide feedback of new features and major changes before releasing it to the general public. This would eliminate a lot of frustration by end users as you have released this change and after a lot of complaints and uproar realised that your new solution does not work any better than the old solution and you have to find another way to implement tax reporting using dual tax codes. With a feedback panel team trying out new features on a development portal all this angst and frustration would not occur in the first place. We had the same problem when you changed the summary page - initially there were a lot of complaints and user frustrations because you released it before it was actually ready to be used by the public. Manager needs a feedback panel of users who can trial new features and report issues encountered, those could be fixed and then when the feature release is actually completed then released to production systems.
For me this is the major issue with the whole approach to manager’s data model. It’s an assumption that it is bug free and correct.
You should NEVER ASS-U-ME. In my opinion, you can’t assume, you shouldn’t assume. That’s where the danger lies. Because when it does change, what changed? The program can’t tell you.
I just thought I would do a cursory search of the internet to see how other packages deal with this issue and it’s interesting to note that with an alternative product under their pre-“live with the ATO version” of the BAS submission that once you completed the worksheet and reviewed it, you “published” it internally to the system which retains a non-editable version. From their documentation:
Click Save or Publish . The BAS group of reports is saved to the Published tab where you can export it to various formats.
Published amounts are used on your GST Reconciliation report as ‘filed’ amounts.
This helps identify any changes in the period after lodgement.
You can’t edit published statements, but you can run them again by choosing the same dates.
I think that demonstrates a couple of key components but essentially that once published any user initiated changes to the period, (as well as any programatic change), is going to be evident when a second report is run and a comparison made. I don’t want to suggest that is a gold standard, but it’s certainly a lot better than assuming there are no errors. Also, this approach protects the user from themselves and that once a report is generated it’s “published” internally to the system in a static form that can’t be edited. I think that is key, because as @tut pointed out initially, and as is the case with manager, all those reports are dynamically generated. I personally do like that, however, my thinking with this proposal is that some reports should be static, or have a static option. They could even just be the entire PDF, it doesn’t have to be HTML at all. @dalacor’s idea of keeping it as data in meta data is also sufficient, but in reality PDFs is probably the best evidence.
Anyway, I think I have made my point and position clear on this, we will simply have to agree to disagree on the importance of this proposal.
No, not at all. I think it was in the other thread I mentioned it, but if the data is wrong, then it does need to be corrected, I don’t have any issue with that. I needs to be corrected. it has to be, it’s not optional. and I do apologise if it came across that way. My issue and point is that if the values for a given report snapshot are changed, there should be some mechanism to highlight that to the user.
Program enhancements and improvements are happening all the time, I’m not saying they should’t happen. I’m not saying that user input is absolutely required. But the user should be notified if key values that are present in one version of manager are different in another version. eg, if your sales figure or tax liabilities appear different in the P&L in 20.5.17 to the ones in 20.8.38, there should be some mechanism to set off some alarm bells. I have grossly cut my COA down to try and make this easier to see in the future
I’m sure they are well considered, and I don’t think he does them on a whim. He might miss some of the considerations that we as users give aspects of the program, elements that we consider important because they would assist us in our workflow, but that is not manager’s core functionality and I generally accept that.
That is the whole, and only intent of my suggestion, although it has the added bonus of protecting the user as well! If the user modifies data that is present in a Report Snapshot, it could be brought to their attention as well.
I started with that idea at one point myself, but I removed it from the original because although the meta data in the history would almost always suffice, if the template for the report changed, or the calculation method for any of that metadata changed, then we would be back at square one. I do like it, but I just don’t think it’s as validating.
and I would add B&L and BS reports too
That’s a good articulation of basically what I was trying to get across at one point, but I did lose my way. Basically I get all flustered and frustrated because my numbers don’t match here and there and I don’t know if it’s me, my accountant transposing things incorrectly, or it’s manager, or what?
My “idea” would have a record that between updates would allow discrepancies between versions to be highlighted and noted, in much the same way bank reconciliations are done. Although I manually do my reconciliations, the intent is that those reports that are snapshotted have the static entries recorded for the various “key” features of your reports, like balances of the COA on a given date, or fields of the BS and P&L, that way when the program detects that one of those values have changed, it can generate a “Snapshot Report Reconciliation Fail” or whatever. This proposal is more about manager storing the data and ensuring integrity, but as exampled by the above alternative competing product, it doesn’t need to be, it could just be a PDF lodged internally. I would give much preference to manager maintaining those static reports, just as an added layer of protection for the users.
Users could manually do this now, print off each and every key document and compare them before and after, but that would be a very long and laborious project. TBH, I’m sure that it may even be possible to use the API and have python scripts do that checking for us. In essence, I suppose it doesn’t even need to be manager that does it.
That is something I can consider. I will give that some more thought.
I have removed almost all my “hacks”, there’s just some minor inventory stuff to get rid of, but that is ancient and so I probably won’t bother. I have tried in as far as possible and as best as I know to do everything now as is intended. I do think I have a couple of things I enter sales invoices for that could be a payment and also the other way, there are a couple of payments that probably should be sales invoices. I have been slowly moving everything (in as far as possible) out of suppliers and into payments.
I do believe that and I suggest (that especially in my case), it’s never in an attempt to “break” manager or to really extend it beyond it’s capabilities, I do it because I believe that the solution I came to was close enough for my use case, and I suspect that others would be like that too. It’s not that we intentionally use wrong accounts or inappropriate tax codes for any reason other than we are trying as best as possible to get all our ducks in a row.
No problem and it’s just a mis-interpretation of what I wrote there. On this update it didn’t change previous BAS, when I mentioned previous BAS’ changing I was talking about when we initially identified “the bug” (last year). It was at that time it messed with the tax amounts on the BAS because they weren’t being calculated because of the zero balances. It was a known problem for some time and I appreciate the fact you promoted it up the ladder because it would have involved a lot of back pedaling to fix all that. It was those previous BAS that changed and that was hinted at when I said
On this occasion, how it came to my attention was because it did change the data, but not on the BAS — on my P&L. From memory, it was specifically to do with inventory management and GST adjustments and again these blasted zero-value/balanced invoices.
I didn’t mean to suggest this bug this year changed BAS statements, I have no idea on that.
And yes, you did fix it very quickly and I am very appreciative of that as I was meeting with the accountant later that day.
At any rate, I can see my suggestion has not been received very well, and the emphasis on validating manager’s output over time is on me. I do realise now, and this creates a sickly feeling in my stomach, that I have been remiss by not keeping PDF copies of those reports. (But yes, I have the backups, but I have no idea what version of manager made each backup, because although I advocate it now, I did not save copies of manager with each copy of the backup. I will see how your allowing of multiple copies to be open pans out in the future).
I will try and let this die a peaceful death, and thanks for putting up with me
OK, well that’s Xero - you can mention names of other accounting software packages. And as you know, they are progressively getting rid of this feature. How they handle it the new way?
Same as Manager. They don’t.
If your figures change they say:
Re-lodge activity statement with new figures outside of their software
or make reversing journal entry to fix your past figures so they are consistent with what you have lodged
This is what I recommend as well.
I suspect the reason Xero is getting rid of this feature is because it doesn’t really answer the question “Why”. Why did the figures change? That’s the most important question.
And I think in Manager, we can leverage History to answer that question. Manager could examine History and find all user actions which has affected general ledger for certain period after certain date.
So I do recongnize the problem. If accountant is working on their client’s file once a year and the historical figures change, the accountant wants to know why did they change. What did their client do during the year which changed the historical figures and take appropriate action. This where History data could play major role.
Report snapshots would only say - yes… the figures have changed. Thank you very much. But why? That’s the big question.
History would give you “where to look”
Snapshots would alert you to the fact you “need to look”
I’m only trying to avoid having problems in the future, being pro active now to changes as they happen and reduce the urgency of trying to sift through a year of history at the last possible moment.
If “why” is the big question, then wouldn’t narrowing down when a problem occurred (by virtue of the snapshot being different to today) be better than rolling back through a year of history? (Focusing on when and how)
The issue is that this can be misleading. If you reorganize your chart of accounts which you often do to adapt better to your business, your report figures can change but those changes are immaterial.
Give you an example. You have single control account for all your inventory items. You decide you want to split your inventory items across multiple control accounts. Your balance sheet will end up with different figures as you are breaking one big account into smaller ones but those changes in figures are not material.
For me the use case is mostly to readily confirm no net changes apply to documentation I have sent to other parties, such as invoices, group certificate, payslips, BAS, Tax (profit & Loss statement, Balance sheet).
I agree when net changes have occurred then Managers detail history feature is the best way to know why. However the “solution” will depend on the details of why the values are different, for example if the old data was wrong for any reason, then ensuring the third party is now working from current values and has discarded the old would be the required action. Together with retaining access to prior correspondence.
The test needs to be simple so it is independent of changes by users, settings, and program code.
A possible implementation is to add view screen snap shots to key Manager screens.
When the user prints, emails or saves from that view screen, the output text and numbers are saved. A simple format should be used that supports easy later comparison, a text file is probably appropriate. I used “Output metadata” above to try and describe this but that’s probably a miss leading description.
When a user later looks at the view screen history, the historical number and text are shown but any values which are different to the current dynamic report are highlighted.
I agree. The changes the user intends would show as changed. But ideally the things which have not changed would also show as not changed such as the totals
Now we are talking about account reconciliation. I think this idea floated on the forum a while back.
For example, you have reconciled your tax account and the figure as per certain date should be $5,000. You want to see if this account balance becomes unreconciled. Just like it works currently for bank accounts.
You are not exactly interested in report matching exactly because layout changes will happen and are imaterial, but you want to ensure specific account balances do not change.
I think that accounts reconciliation is the way to go. Also he same tool can be used to extend the actual budgeting solution since it will work both on BS and on P&L.
