Ever since I was a child and can remember, I have always believed in the practice of backtesting my predictions for their accuracy. If I thought something would happen, did it actually happen or not? A simple and robust example of this is how I used to first predict my exam results and then backtest them to validate the accuracy of my predictions.
During my school years, I always remember crying and complaining to my mother that I simply can’t understand why I did not get the 1st, 2nd, or 3rd position in the class when I had answered everything correctly in the exams. I always used to get the 4th position. My mother always responded encouragingly by saying that the 4th is where the real 1st starts from because the parents of the children who were getting the 1st, 2nd, or 3rd positions were paying the school hefty donations. My mother always told me to watch out for the school final exams (called the 10th board exam in India or GCSE level in the UK) which were conducted and marked by the national examination board and not the school authorities.
Because my mother was so sure and for so many years had told me to watch out for the 10th board exams, when they finally happened, straight after every exam when I reached home, I decided to calculate what I thought I would score in that exam’s question paper. For those that I could totally certainly calculate, like Mathematics, there was no need for any confidence interval and it would be just one number, however, for subjects belonging to humanities like literature, I would remember to add a range on the exam paper. I kept those exam papers securely so that when the results were declared (around three months later) I had them handy to cross-check.
Because of this practice of marking my own paper at home, I remember crying profusely to my mother, when my prediction for the mathematics paper was that I would score a 99 out of 100 instead of the full score. Now as a mother myself, I can imagine how elated my mother would have been hearing me say that I would score a 99 on my math paper, but for me, the world had crumbled down as I had missed out on responding to a 1-mark question on the exam.
Then the actual results were declared which finally allowed me to backtest the robustness of my predictions. And exactly how my mother had believed, I was the 1st position holder from the school with all those students who used to get the top positions until that year nowhere close to me in ranking. I had achieved a 99 in Mathematics and all the other subjects were within the range I had predicted.
Motivated by the success of my first proper backtesting results, I decided to replicate the approach two years later in my 12th board exams (or A levels in the UK). Same as earlier, after every exam when I reached home, I would go through the entire question paper, remember how I had answered them, mark each question, and then sum it up to note down the total on the front sheet of the exam’s question paper. Then I would neatly file all those question papers so that I had them available to refer to when the results were declared in around three months’ time.
When my results for the 12th board exams were announced, I had topped the entire state in Commerce stream. Everybody including my parents had expected me to do well, but the quantum of success surprised everybody. However, I was not so happy because my backtesting had failed for English literature. The marks I was awarded were remarkably lower than the lower bound of the range I had predicted. I had so much confidence in my predictions through the backtesting from the 10th exams, that I decided to request my school’s headmaster to approach the State Board for a re-checking of my paper.
Because my headmaster used to take our English classes himself, and he knew how good I was in literature, he agreed with the need for a recheck. To ensure it happens properly, he decided to go to Kolkata himself and spoke to the officers on the Board. The response he got from the Board surprised us all – the marks I had scored in the 12th board exams in Commerce stream were the highest recorded marks in the Board’s history, and if I was given any more in English than what I had already received, I would have gone beyond what a science student scores with many more technical subjects. They did not want this and hence penalizing a qualitative subject like literature was the only option. We did not argue anymore. My confidence in my backtesting was restored.
In my three years of graduation in Mathematics with Honors, I continued to follow my approach. At the end of every year, if the backtesting showed a deviation of a great degree than I was willing to accept, I started requesting the university to recheck my answer sheets. 100% of the times I had asked for a recheck, my scores were revised upwards. My approach built over the years had started rewarding me.
Once I finished my education and started working, I did not want to forget about this tried and tested formula I had created. So, I adapted it to work without an exam setting. At the end of every year, I sat down to create my resolutions for the next year (or goals) and also backtest if I had achieved what I said I will achieve the year that had finished.
If I have to summarise my annual backtesting performance using this approach, it mostly shows that I have normally achieved the goals I had set for myself which were clear and measurable, and not achieved those which were vague or totally unrealistic.
And of course, no surprises for guessing that the backtesting results for 2020 and 2021 so far, being the years of the Pandemic, show that just like most of the forecasting models used in the corporate sector built on vast historical data have not been able to predict accurately such a once in many lifetimes scenario, my own prediction model, built successfully and refined over the years since school, has also failed to be accurate. The prediction exercise for 2022 is due soon, and I sincerely hope that the end of 2022 will show a more accurate backtesting performance.
Comments