I must say it's nice for you to pop along here now and again to see if anything is happening. I have news for you and a lesson to be learned, but you might not like it. I certainly didn't.
This message is aimed at those people who spend hours on top of hours back testing their system whether by hand or by a computer model. so in other words any trader ever trying to make a trading system.
The lesson is that most likely your system has a missing element - read this post thoroughly and you'll find out what it is.
The system we knew as Grail is no longer in use. It is no longer productive. That does not mean that it will never work again, but that it no longer gives the right return for the amount of risk taken.
In December last year (2006) grail had a drawdown of around 50%. Bear in mind that at it's height the system had made 1000% in a year and had taken 10k into 100k - a significant achievement I'm sure you will all agree.
The December drawdown was a crushing blow. I was in New York at the time having a holiday and it totally spoilt that!. Of course the day that we as a team decided to reduce the stakes was the day the system decided to turn around. we changed from £118 per pip to £50.
thereafter followed a 17 day winning streak. to the point that had we remained on our original staking plan, new account highs would have been made on day 14. amazing eh.
so think about this, OK, you reduced stake and the system did exactly what you thought it would do. it returned to form. We did not do what most would and go back to large stakes, we took the hit and moved on from our current position.
December though should have rang warning bells. The fact that the drawdown was so severe and sudden - within 18 trading days from a new high at the end of November we were almost on our knees. Clearly the system was prodding us in the ribs saying "oi, take notice twat" we didn't.
we continued to trade the system - January did amazing numbers but then that was it.
Feb lost, March lost, April lost and this time it was a drawdown of almost 60% from the November highs when we finally gave up the ghost and ceased trading Grail. From over 100k i had around 38k left. - gone in less than six weeks.
When you think about it, the system was a success. It made 38k from 10k and in addition to this we traded it on another account and took over 40k in 'wages' from 10k of that initial winnings. so not bad.
we are still tracking the system and it's lost and lost and lost - if we would have continued it would without a doubt have blown the account totally. The bottom of the drawdown was 1082 pips, met a couple of weeks ago.
So what went wrong and what can we learn from this to move on. More importantly how can it help you.
The Testing
When we first devised the system and tested it we hand tested almost three years of data. - that was around 600 trades. Would you consider that adequate? remember machine and HAND tested?
It sounds like plenty of testing doesn't it and believe me it was hard work. we changed this, changed that, aware all of the time that we must not curve fit. Checking each trade manually on a one minute chart.
Our final test prior to live ran from May 1st 2003 to September 1st 2005 (remember we started trading it in October 2005)
Below is a snapshot of the equity curve of the test - good eh?
Well, we thought it was!
In fact we thought it good enough through our months of testing and refining that we began trading it in October 2005.
The tales of our experience from here are in this very blog if you go back far enough - it's a fascinating read and you will enjoy it.
Live Trading
to cut a long story short, one year in and all is well. The system performed not just close but EXACTLY how we expected it to, hitting 1000% within just 4 weeks of the anticipated date of October 31 2006.
Imagine the elation knowing that all you have to do is repeat what you've already done for a year, for the next year and you would be sat on a cool million. At this point we were all 100% committed to this in the 100% belief that this system works. We even had investment funds now trading the system. Millions were being traded with it.
The test data now had an extra year of trades but this time REAL trades and not test trades. The test was now over 800 trades with 20% of them done in the real market.
have a look at this chart - the test matched reality exactly - this was may 03 to November 06
The Fall
We know the story from here, just go back a few posts and you'll see what happened. The system failed. But why did it fail and what did we do wrong and not include?
Ceasing to trade the system was one of the hardest decisions I have ever made as a trader. It didn't mean just my dreams evaporating, but those of my team, My wife and the capital of my investors. Not an easy decision and one that still makes me shudder to this day.
As a team and as human beings we went through what i can only explain was a grieving process. we just didn't know what to do or where to turn. we had got so used to mechanical trading that our discretionary skills has dried up and withered so we had no backup plan.
You may laugh at this but this grieving process is only just coming to an end and was one of the most painful times in my life. Ive always been successful in whatever I've done and when I've set a goal I've reached it. This time I did not.
The reason i have the strength to continue this blog is because I am now back on the road and once again trading very well but i learned so much from the experience, in fact some will laugh, but I'm a changed person because of it.
before we move on, allow me to share with you the equity curve that the test returns when ran today. - the high peak you can see is November 2006 - the second lower peak is January ran with lower staking and you can see what happens from there - direct evidence i suppose that trends exist in every part of trading including equity curves!
The Analysis
So, lets look at what went wrong.
Those of you who backtest systems wont like this. Most think that a few months or just a year is long enough to test. It isn't.
Our test, even almost 4 years of it with 800 trades of which 20% were real - was not enough.
Why? because our test did not include all market conditions.
When the grief set in we started to dissect the system and find out what went wrong, why the system failed and what we could have done to mitigate.
After a lot of talk and soul searching we looked at the date we used to test - it was fine but when we looked at a monthly chart, it was strikingly obvious.
Here is the chart.
Left to right, the first line is the start of our testing period, May 2003. the market is fine and normal. 2003 was not a great year in the test though acceptable and you can see why it did not fair so well as the first five months of the test were up and down but still had nice ranges.
The second line on the chart represents October 2005 - when we started trading the system. so that bit of data between the first two lines are our test period - not a lot really is it? does it really look like it has all market conditions in it? no, it's a nice trending market most of the time.
so line 3 is November 2006 - the peak of our grail accounts when we hit 100k and 1000% on account - a nice trending 14 months i think you will agree.
Now look what happens between November 06 (line 3) and April 07 (line 4).
Four Doji bars in a row. a severe sign of a constricting, indecisive market if ever i saw it.
No trends, just spikes up and spikes down - the daily action mimicked this spikes up then down, spikes down then up.
Grail was an intraday trend following system. during this time there were no trends, just a hell of a lot of reversals.
During this time those in the forums were developing systems such as Firebird and grid trading systems - they did really well during this time because they were mean reversion systems - they traded counter trend to moves looking to profit on the retraces - and there were plenty of them. Systems similar in nature to Grail such as Hans 123 was also suffering and also died during this time.
The Firebird guys tested like a few months and thought that covered all market conditions because they had lots of trades a day- clearly they were only testing these monthly doji bars and when the market did start to trend for as little as a few days they got burned badly only for it to turn in their direction the very day after they took losses.
anyway, as you can see, those market conditions did not exist within our test period. In fact we never had more than a single doji in a row, let alone four of them.
And that's what killed Grail, our test data just was not enough and did not cover all market conditions.
Before i move away from this, just think for a minute and look at that chart.
Every month someone is saying "the markets have changed, the volatility will not return" - well look at the chart - it stretches all the way back to 2001 - can you see any real market change there? - i know i can't, just a trending market with a few area's of consolidation is all.
So, the questions arise like, well, if the test data had those kind of periods in then you may have never traded the system, and that's true - chances are we would've either not gone forward or we would have played it much less aggressive. The thing is we do not have a crystal ball and we don't know what we would have done.
The fact remains that our test was not enough and for your information, for the test to have those conditions in it would have had to have been an nine year test and even then the four doji's in a row were much smaller in 1996.
Lessons
The lesson here and my message to you is that no matter how much you test, no matter how far you go back there will always be a market condition that your test does not have. Next time it could be 7 doji's together or even 17. we don't know.
The market ranges have also got much much smaller - in 2004 the range for cable was almost 170 pips a day 05 was around 150, 06 was 140ish and this year has barely managed 110. but go back further to the early 90's and you'll see ranges that will make you sweat just by looking.
Grail was a trend system and without a trend she died.
So, those of you designing systems are now saying 'well, what the hell can i do? - i have to rely on the fact that the past repeats it's self otherwise i might as well flip a coin'
My answer is that you need do nothing different apart from add one thing extra to your system that we did not.
in the testing phase our system worked for four years solid - two more years and i would never have traded again because i would have had 10 million quid in the bank. we were unlucky with market conditions and you may be luckier.
LTCM (Long Term Capital Management) had around five years of amazing trading. they were the Wall Street kings for years. but then market conditions arose which weren't in their test period.
here were a bunch of professors from MIT. they were the best of the best and made more money than anyone else before them. Yet they went bang big style bringing the financial world to it's knees for billions.
One thing is for sure. no matter how good your system appears to be, no matter how much money it has already made and no matter how much confidence and testing you put into it - sooner or later a market condition will surface which you haven't seen before. we never know what is around the corner, you could run the gauntlet and have 10 years or more of great results. but sooner or later the market will throw off a wild one.
What's needed is something that traders hate to have to think about.
Long Term Capital Management didn't have one when they had to be rescued by the Federal bank for Billions. Nick Leeson didn't have one when he broke Barings Bank, the oldest bank in the world. and Soultrader and his team didn't have one when grail failed.
What they didnt have was a method of identifying when the system failed and a plan of what to do when that happened.
I have no doubt that grail Will become profitable again when the market returns to it's groove. In fact it's starting to look like the market ranges are coming back now. for months the 20 day average has been below 70 on cable. recently it's got above 100 for the first time in months. Grail will work again and i might have a punt again when it does - there could be another four years!.
System designers need to add one more thing into their arsenal. They ask these questions when designing a system
- When to enter a trade
- How much to trade
- Where to place a stop
- Where to exit a trade
they are missing part 5 and part 6
5. How to identify when the market conditions cause the system to fail.
6. What to do when it's failed.
There is also something else that we have learned about our system design that i'd like to sharte with you.
Grail was an intraday trend following system. it traded every day, when the intraday trends disappeared so did grails profits.
Grail, with certain parameter changes would work on most currency pairs, in fact, most liquid markets.
Grail was designed around a fundamental truth - trends exist. The fact that for the past six months that have not existed intraday does not mean they aren't there on longer timeframes.
Our trading is still trend based, but no longer tied to a specific time period and no longer tied to a single pair. I now trade trends over the longer term and on any currency you can name. This new method of trading is better than Grail and so good things have come of this.
My advice to you is that when you design a system make it a system based on a solid market principle. and don't limit yourself - if it's good it will work on ANY currency pair.
The End Is Nigh!
This grailtrading blog has been here since 2005. I hope you've all enjoyed it. If all this blog does is make you implement parts 5 and 6 into your trading systems then it stands here for something and leaves a legacy.
Since grail we have moved on as a team. We are scarred and battered but not beaten. The true mark of life's winners is that they do not give in.
The whole experience has changed me as a person. I'm still an arrogant sod but i have tasted real humility and have been just about as low as you can get emotionally. I'm not going to say that I have come out stronger because it's not true. I've come out more cautious than ever before and very cynical - two traits of a trader that are never publicised in the books you read but two which will enable me to move forward with my chosen career in safety and with a degree of humility.
Until you've been there you cannot possibly imagine what me and my team have been through from the heady heights of amazing returns to the gutter lows of complete despair.
This is the final post for this blog and i really hope those of you who have shared this journey have learned from it. If you are a currency trader and are finding this for the very first time I strongly suggest that you read this blog from the very beginning. feel the highs and lows with us as you reach this final step.
Start reading here and use the archives
And good luck to all for the future.
The Team:
Soultrader
Keres
Angiefx
Trevor Automan
Madisonfx
Mike Whisper
Fluty
