What went wrong?

The Million Pound Experiment - The Grail system.





I must say it's nice for you to pop along here now and again to see if anything is happening. I have news for you and a lesson to be learned, but you might not like it. I certainly didn't.

This message is aimed at those people who spend hours on top of hours back testing their system whether by hand or by a computer model. so in other words any trader ever trying to make a trading system.

The lesson is that most likely your system has a missing element - read this post thoroughly and you'll find out what it is.

The system we knew as Grail is no longer in use. It is no longer productive. That does not mean that it will never work again, but that it no longer gives the right return for the amount of risk taken.

In December last year (2006) grail had a drawdown of around 50%. Bear in mind that at it's height the system had made 1000% in a year and had taken 10k into 100k - a significant achievement I'm sure you will all agree.

The December drawdown was a crushing blow. I was in New York at the time having a holiday and it totally spoilt that!. Of course the day that we as a team decided to reduce the stakes was the day the system decided to turn around. we changed from £118 per pip to £50.

thereafter followed a 17 day winning streak. to the point that had we remained on our original staking plan, new account highs would have been made on day 14. amazing eh.

so think about this, OK, you reduced stake and the system did exactly what you thought it would do. it returned to form. We did not do what most would and go back to large stakes, we took the hit and moved on from our current position.

December though should have rang warning bells. The fact that the drawdown was so severe and sudden - within 18 trading days from a new high at the end of November we were almost on our knees. Clearly the system was prodding us in the ribs saying "oi, take notice twat" we didn't.

we continued to trade the system - January did amazing numbers but then that was it.

Feb lost, March lost, April lost and this time it was a drawdown of almost 60% from the November highs when we finally gave up the ghost and ceased trading Grail. From over 100k i had around 38k left. - gone in less than six weeks.

When you think about it, the system was a success. It made 38k from 10k and in addition to this we traded it on another account and took over 40k in 'wages' from 10k of that initial winnings. so not bad.

we are still tracking the system and it's lost and lost and lost - if we would have continued it would without a doubt have blown the account totally. The bottom of the drawdown was 1082 pips, met a couple of weeks ago.

So what went wrong and what can we learn from this to move on. More importantly how can it help you.

The Testing

When we first devised the system and tested it we hand tested almost three years of data. - that was around 600 trades. Would you consider that adequate? remember machine and HAND tested?

It sounds like plenty of testing doesn't it and believe me it was hard work. we changed this, changed that, aware all of the time that we must not curve fit. Checking each trade manually on a one minute chart.

Our final test prior to live ran from May 1st 2003 to September 1st 2005 (remember we started trading it in October 2005)

Below is a snapshot of the equity curve of the test - good eh?


Well, we thought it was!




In fact we thought it good enough through our months of testing and refining that we began trading it in October 2005.

The tales of our experience from here are in this very blog if you go back far enough - it's a fascinating read and you will enjoy it.

Live Trading

to cut a long story short, one year in and all is well. The system performed not just close but EXACTLY how we expected it to, hitting 1000% within just 4 weeks of the anticipated date of October 31 2006.

Imagine the elation knowing that all you have to do is repeat what you've already done for a year, for the next year and you would be sat on a cool million. At this point we were all 100% committed to this in the 100% belief that this system works. We even had investment funds now trading the system. Millions were being traded with it.

The test data now had an extra year of trades but this time REAL trades and not test trades. The test was now over 800 trades with 20% of them done in the real market.

have a look at this chart - the test matched reality exactly - this was may 03 to November 06



The Fall

We know the story from here, just go back a few posts and you'll see what happened. The system failed. But why did it fail and what did we do wrong and not include?

Ceasing to trade the system was one of the hardest decisions I have ever made as a trader. It didn't mean just my dreams evaporating, but those of my team, My wife and the capital of my investors. Not an easy decision and one that still makes me shudder to this day.

As a team and as human beings we went through what i can only explain was a grieving process. we just didn't know what to do or where to turn. we had got so used to mechanical trading that our discretionary skills has dried up and withered so we had no backup plan.

You may laugh at this but this grieving process is only just coming to an end and was one of the most painful times in my life. Ive always been successful in whatever I've done and when I've set a goal I've reached it. This time I did not.

The reason i have the strength to continue this blog is because I am now back on the road and once again trading very well but i learned so much from the experience, in fact some will laugh, but I'm a changed person because of it.

before we move on, allow me to share with you the equity curve that the test returns when ran today. - the high peak you can see is November 2006 - the second lower peak is January ran with lower staking and you can see what happens from there - direct evidence i suppose that trends exist in every part of trading including equity curves!


The Analysis

So, lets look at what went wrong.

Those of you who backtest systems wont like this. Most think that a few months or just a year is long enough to test. It isn't.

Our test, even almost 4 years of it with 800 trades of which 20% were real - was not enough.

Why? because our test did not include all market conditions.

When the grief set in we started to dissect the system and find out what went wrong, why the system failed and what we could have done to mitigate.

After a lot of talk and soul searching we looked at the date we used to test - it was fine but when we looked at a monthly chart, it was strikingly obvious.

Here is the chart.


Left to right, the first line is the start of our testing period, May 2003. the market is fine and normal. 2003 was not a great year in the test though acceptable and you can see why it did not fair so well as the first five months of the test were up and down but still had nice ranges.

The second line on the chart represents October 2005 - when we started trading the system. so that bit of data between the first two lines are our test period - not a lot really is it? does it really look like it has all market conditions in it? no, it's a nice trending market most of the time.

so line 3 is November 2006 - the peak of our grail accounts when we hit 100k and 1000% on account - a nice trending 14 months i think you will agree.

Now look what happens between November 06 (line 3) and April 07 (line 4).

Four Doji bars in a row. a severe sign of a constricting, indecisive market if ever i saw it.

No trends, just spikes up and spikes down - the daily action mimicked this spikes up then down, spikes down then up.

Grail was an intraday trend following system. during this time there were no trends, just a hell of a lot of reversals.

During this time those in the forums were developing systems such as Firebird and grid trading systems - they did really well during this time because they were mean reversion systems - they traded counter trend to moves looking to profit on the retraces - and there were plenty of them. Systems similar in nature to Grail such as Hans 123 was also suffering and also died during this time.

The Firebird guys tested like a few months and thought that covered all market conditions because they had lots of trades a day- clearly they were only testing these monthly doji bars and when the market did start to trend for as little as a few days they got burned badly only for it to turn in their direction the very day after they took losses.

anyway, as you can see, those market conditions did not exist within our test period. In fact we never had more than a single doji in a row, let alone four of them.

And that's what killed Grail, our test data just was not enough and did not cover all market conditions.

Before i move away from this, just think for a minute and look at that chart.

Every month someone is saying "the markets have changed, the volatility will not return" - well look at the chart - it stretches all the way back to 2001 - can you see any real market change there? - i know i can't, just a trending market with a few area's of consolidation is all.

So, the questions arise like, well, if the test data had those kind of periods in then you may have never traded the system, and that's true - chances are we would've either not gone forward or we would have played it much less aggressive. The thing is we do not have a crystal ball and we don't know what we would have done.

The fact remains that our test was not enough and for your information, for the test to have those conditions in it would have had to have been an nine year test and even then the four doji's in a row were much smaller in 1996.

Lessons

The lesson here and my message to you is that no matter how much you test, no matter how far you go back there will always be a market condition that your test does not have. Next time it could be 7 doji's together or even 17. we don't know.

The market ranges have also got much much smaller - in 2004 the range for cable was almost 170 pips a day 05 was around 150, 06 was 140ish and this year has barely managed 110. but go back further to the early 90's and you'll see ranges that will make you sweat just by looking.

Grail was a trend system and without a trend she died.

So, those of you designing systems are now saying 'well, what the hell can i do? - i have to rely on the fact that the past repeats it's self otherwise i might as well flip a coin'

My answer is that you need do nothing different apart from add one thing extra to your system that we did not.

in the testing phase our system worked for four years solid - two more years and i would never have traded again because i would have had 10 million quid in the bank. we were unlucky with market conditions and you may be luckier.

LTCM (Long Term Capital Management) had around five years of amazing trading. they were the Wall Street kings for years. but then market conditions arose which weren't in their test period.

here were a bunch of professors from MIT. they were the best of the best and made more money than anyone else before them. Yet they went bang big style bringing the financial world to it's knees for billions.

One thing is for sure. no matter how good your system appears to be, no matter how much money it has already made and no matter how much confidence and testing you put into it - sooner or later a market condition will surface which you haven't seen before. we never know what is around the corner, you could run the gauntlet and have 10 years or more of great results. but sooner or later the market will throw off a wild one.

What's needed is something that traders hate to have to think about.

Long Term Capital Management didn't have one when they had to be rescued by the Federal bank for Billions. Nick Leeson didn't have one when he broke Barings Bank, the oldest bank in the world. and Soultrader and his team didn't have one when grail failed.

What they didnt have was a method of identifying when the system failed and a plan of what to do when that happened.

I have no doubt that grail Will become profitable again when the market returns to it's groove. In fact it's starting to look like the market ranges are coming back now. for months the 20 day average has been below 70 on cable. recently it's got above 100 for the first time in months. Grail will work again and i might have a punt again when it does - there could be another four years!.

System designers need to add one more thing into their arsenal. They ask these questions when designing a system

1. When to enter a trade
2. How much to trade
3. Where to place a stop
4. Where to exit a trade

they are missing part 5 and part 6

5. How to identify when the market conditions cause the system to fail.
6. What to do when it's failed.


There is also something else that we have learned about our system design that i'd like to sharte with you.

Grail was an intraday trend following system. it traded every day, when the intraday trends disappeared so did grails profits.

Grail, with certain parameter changes would work on most currency pairs, in fact, most liquid markets.

Grail was designed around a fundamental truth - trends exist. The fact that for the past six months that have not existed intraday does not mean they aren't there on longer timeframes.

Our trading is still trend based, but no longer tied to a specific time period and no longer tied to a single pair. I now trade trends over the longer term and on any currency you can name. This new method of trading is better than Grail and so good things have come of this.

My advice to you is that when you design a system make it a system based on a solid market principle. and don't limit yourself - if it's good it will work on ANY currency pair.

The End Is Nigh!

This grailtrading blog has been here since 2005. I hope you've all enjoyed it. If all this blog does is make you implement parts 5 and 6 into your trading systems then it stands here for something and leaves a legacy.

Since grail we have moved on as a team. We are scarred and battered but not beaten. The true mark of life's winners is that they do not give in.

The whole experience has changed me as a person. I'm still an arrogant sod but i have tasted real humility and have been just about as low as you can get emotionally. I'm not going to say that I have come out stronger because it's not true. I've come out more cautious than ever before and very cynical - two traits of a trader that are never publicised in the books you read but two which will enable me to move forward with my chosen career in safety and with a degree of humility.

Until you've been there you cannot possibly imagine what me and my team have been through from the heady heights of amazing returns to the gutter lows of complete despair.

This is the final post for this blog and i really hope those of you who have shared this journey have learned from it. If you are a currency trader and are finding this for the very first time I strongly suggest that you read this blog from the very beginning. feel the highs and lows with us as you reach this final step.

Start reading here and use the archives

And good luck to all for the future.

The Team:

Soultrader
Keres
Angiefx
Trevor Automan
Madisonfx
Mike Whisper
Fluty

Carry an Umbrella in your head.

Ok, time to reveal what happened at Christmas then. I will tell you what happened but will not touch on the legal stuff - sit back, relax and have a giggle at my expense....

You see over a year ago we devised this system we now trade called grail.

Our backtests (of 3 years) showed us what the method would do and what we should expect. How often we would have drawdowns, how much we should make month by month and what sort of equity curve we should have.

So we began trading it - this story is embedded in the pages of this site and if you want to look at the whole journey then please have a look around and you may get into the same feelings we did.

Month by month we made money. Sure we had a losing month here and there and we had drawdowns - but the whole experience of trading the method with real money was very much as expected. In fact it was almost exactly as expected.

Our journey went from initial excitement of starting off, to a nice steady feeling of expectation. Of course all this time we were not just sat here watching charts - all the time we were testing and verifying what our previous tests had given us.

months and months of tests, all saying the same thing. whenever we had a new drawdown we just looked at our tests to see if this was anything new and, as it clearly was not new, we just kept on sailing on. Day by day.

We began to take this success for granted - believe me, it's hard not to when week in, week out you make money and everything goes exactly as you have planned.

So, what we did was to start looking at how we could make more money from this strategy. - if you look back in the posts you'll see where we did this.

What we did would almost bring us and our account to our knees.

At the time we had a different account called our 'income account' - on that account we still traded grail allbeit with a different leverage. but in that account we had equity targets - so when the accounts reached a certain level we could take out our winnings to live on etc.

This meant that i could give up discretionary trading (something i've always managed to do but hated) and just let this golden method make my wages too. The idea was that having two accounts meant that we could let the main account grow and use the income account to live on. Makes sense yes?

Anyway, the decisions we made next was to be the humble pie maker of a decision.

We sat with spreadsheets and worked out how much of the account was 'safe money'. after some serious calculations and allowing for error we decided that 25% of the account was always 'safe money' - in other words, no matter how bad a drawdown we got that 25% would always be there and would manage to remain untouched. So, we then decided that at certain equity highs we could take out 25% - makes sense doesnt it that if 25% is never going to be touched by a drawdown, then why leave it in there? - well, that was our thinking anyway.

So at an equity high of £40,000 we happily took out £10,000 - which was our initial stake into the fund. So what do we do with this 25% of 'safe money' - do we put it in a building society? do we invest it into national savings ora tax free ISA account?

No, what we did was put it in the place that we believed would give the best return on our money - we put it in the income account so we could trade it.

Everything is hunky dory, swimming along, making money in the main account and making money in the income account - taking our winnings and enjoying the results.

So, what we were now doing was trading two accounts - the income account at a fixed leverage, never moving the stake and the main account which had a rather unique money management system.

The main account staking plan was now being traded on what we called 'virtual' funds - so at £80,000 when we took out the second £10,000 we still traded the account as if it was still at £80,000 when in reality it was at £60,000 with the other £20,000 in the income account but still available.

The income account generated over £40,000 in winnings during the first twelve months that were taken and spent on living and nice holidays etc.

The account went into drawdown just as the main account did, and pulled back to new profits, just as the main account did. Exactly as expected and all was happy in la-la land.

So, now you have pictures of people enjoying life, making money hand over fist and having fun - the traders dream. After all, what could go wrong?

The aim with Grail was to make 1000% per year - 10k to 100K in year 1 and then 100K to 1000K in year 2 - 10k to 1 million as the title says.

In November 2006, just one month later than expected we hit £100,000 for the main account (£80,000 +£20,000 'safe money') Fantastic! the plan was on track.

Think about what we had done here. We had taken an account, worked out how much money was safe and had effectively ran this account at 100% efficiency. - the method did exactly what it was supposed to do - when it did a drawdown it always came back. ....always.

This still did not change in December - it did come back. However what did come in December was something that we knew could come along any day - but that we had not mentally prepared for.

During our 'pre' grail testing we did see a rather nasty drawdown of 525 pips. at best that is a 40% drawdown on a leveraged account and with our money management we knew we could ride it out. - at least we knew we could technically.

Of course by this time it didn't matter - we were untouchable - this has worked for months and months. Sure, we had seen nasty drawdowns but every time we did it re-enforced the fact that the system was stable

December came and so did our monster drawdown - it did not just come along, it swooped down upon us in a matter of days. On november 30th we were at account highs, and by December 6th, just 4 trading days later we were already 26% down.

On the 12th I was flying to New York nursing a drawdown of 38%.

We had been at this level of drawdown before. What was new was the length of time it took to happen. Usually this depth of drawdown took a good 2 weeks to get there and here we were just seven days from our six figure high with £40,000 less.

But now get this - the account is remember a 'virtual' £60,000 - just £40k in the account with £20 elsewhere right? - nope, because the income account was in an even deeper drawdown. the leverage on income was higher than the main account you see.

You can imagine the way I felt in NYC. I had a system that i believed in so much which had still not reached it's worst drawdown from the past. And I had a team of people with just as much money on the line as me. oh yes, I had a ball in NYC :(

By the 22nd of December, the date of my return to the UK the account was a whopping 48% drawn down.

This was represented by a £48k virtual balance which was not there - the extra £20,000 had gone from the income account which was hovering at the £10,000 level - the amount i originally put in it and the main account was at £28,000.

From a combined account high on November 30th 2006 of £110,000 I had £38,000 left.

Nice close for Christmas was that.

Its important to note at this point that the system STILL had not failed. in fact it could go for another 100 pips to beat the previous drawdown from our tests in 2003.

Many of you may be thinking at this point that the system had failed. but it had not. We had failed the system.

What had failed was the way we had managed the account to 100% efficiency. We had worked out that 25% of the account would be safe and then went on to make that safe money unsafe. basically in our greed what we did was create a fantastic system, but one that if it ever did fail would leave the account totally empty. That's 100% efficiency. Brilliant in theory, devastating in reality. Certainly not my finest hour and i'm sure many of you are enjoying the mental pictures of me squirming under a drawdown. Just remember tho that at least ive got to this position :)

So here we are with a realisation that we are deep shit and if we do see the mother of all drawdowns, we are fubar with nothing.

Well, nothing is a bit of an overstatement, we did take like £40k out of it and we still had £38k left so not bad for a 10k start.

BUT, it was decision time. It was Christmas and we closed at the bottom of our biggest ever drawdown with the thin xmas trading days to look forward to. brilliant (not).

The mistake was clear. It was the fact that we took it for granted that the system would not fail us and in fact it didn't - we failed the system. Taking the money out and then trading that money the same way was very foolish and the realisation of what we had done was very humbling to say the least and still is.

We had a choice - continue with the current stakes and risk that in three trading days we are dead in the water. Or lick our wounds, take the hit and live to trade another day.

Over Christmas we decided on the latter. We drew a line under December 27th and reset the clock. Whatever our balance was at the close of business that day was the new starting point. Our money management would be reset to flat and the money left would be pooled inhto one singular new grail as opposed to income and main growth.

And that's what we did. Of course what happened was bound to happen - the next 12 straight days were winners. if we would have stuck with the stakes we were at then within those 12 days we would have seen new equity highs of over £120,000. but there are lots of shoulda woulda coulda tales in this game. Shit happens and we sat in it big style.

We learned a very hard lesson and had to eat some humble pie as im chewing on now to you lot. The lesson is that we never ever take any part of the account for granted - had that extra 20K still been in the account we would have weathered the storm. It wasn't and we didn't.


Since this time we had a 61% month for January and have just had a poor february at - 12% but we move into March with a renewed outlook .

I have also changed my broker strategy. my account is now spread over several brokers rather than just one and whenever I need money, which to be honest aint often, I will dip into it.

After xmas I started trading again with around £38,000 and the last high of the account since then was in the mid $70k's - I took £3,000 for some living expenses (well, a nice shiny new toy actually) and it now leaves me starting March 2007 at around £60,000. It's been a storming start to the year so far so let's hope it continues.

So, that's my story of what happened at Christmas - the season of goodwill :) I hope you've enjoyed the story and that you may learn from it.

This is the true story of what almost brought me to my knees as a trader - I've been trading now for almost 5 years so if you are new in this game or if you are not the lesson is there - never
but never take anything for granted.

Those who know me know that i'm not good at humility and don't take well to making myself look a twat. It's not been easy for me to tell you this story and would have been easier just to close the blog and never return.

However I think that there are many out there who could easily make this same mistake and if I can stop them doing this then maybe one day they will thank me and tip their hat to the Soultrader and his team.

Finally, and to get give this story a tagline for you to remember - Always remember the trader's golden rule - under every silver lining - there's a fucking huge thick black cloud waiting to pour down on you. Make sure you carry an umbrella in your head.

This is the last post of this blog - The aim continues to make the million, but for now it will remain private

Stop assuming

Hi All, just thought i would drop in a comment here about my blog and the talk going around the web

I don't normally bother commenting but ive seen so many visitors come from sites such as forextsd and moneytec that you are obviously interested. There have been some comments in some forums that need to be addressed

Firstly let me tell you a few facts.

1. I dont have a problem with a 50% drawdown- in fact im happy providing the system does not touch 65%.

2. the bigger the drawdown the better the system works - thats because of a very clever money management strategy which i cannot reveal.

3. at the moment, due to a legal reason, i cannot explain exactly why this last drawdown caused a problem - one thing is for sure it was not the depth of the drawdown - the 50% thing is quite acceptable (though yes, it is painful and i would prefer not to have it!).

Sometime in the near future i will be posting to the blog exactly why it was a problem and what lessons we learned and the actions taken.

Since Christmas eve the system had 12 winning days on the trot - the drawdown was 415 pips and the recovery 515 pips in 2 weeks.

The project to make 1 million from £10k is still a reality and is still alive - the timescale has changed but then this is a long journey and one worth taking - when a pilot in London boards his plane to go to New York he is off course 90% of the time. Making corrections as he goes along and doing the same things over and over again, making small changes here and small changesthere, eventually he reaches his destination.

what i would say to you is don't write this off yet. Many people are talking about the grail experiment and are saying it's over and done - nothing could be further from the truth and all will be revealed as soon as I can.

so knock that smirk off your face right now! (lol)