Former Diplomatic Editor at The Times  Michael Binyon discusses the challenge Isis pose to both the West and Arab world, and whether counter-terrorism strategies are becoming increasingly counter-effective

American bombs rain down on the Islamic State forces attempting to capture the town of Kobani. British troops arrive in Kurdistan to train Peshmerga fighters. Shia militias attack Sunni targets in Baghdad. Iranian troops are sent to reinforce President Assad’s regime, while Turkey mobilises its army to protect the country’s long southern border. And Iraq, the cockpit of the fighting, has descended again into vicious sectarian conflict that may split the country into three warring parts. Is the Middle East now seeing the start of a third Gulf war?

The turmoil and the violence raging through the region seems as confusing as it is dangerous. What began as the Arab Spring has been hijacked by Islamist extremists taking advantage of civil unrest, tribal enmities and religious divisions. The civil war in Syria has already created millions of refugees and taken thousands of lives, and has now spawned a deadly new danger – the emergence of a movement that has surpassed even al-Qaeda for its brutality and ruthless intimidation of its enemies.

The rapid military success of the terrorist movement originally called the Islamic State in Syria and the Levant (Isis, or alternatively ISIL) has thrown the West and the Arab world into confusion. The movement, originating in Iraq and swiftly moving into Syria to confront the Assad regime, has fed on the widespread Sunni disaffection with the Shia sectarianism of the former Maliki government in Iraq as well as the divisions in the opposition to Assad.

Its military victories have challenged everyone in the region. The proclamation of a so-called Caliphate – with the movement now calling itself simply the Islamic State (IS) – calls into question all the borders in the Arab world since the division of the former Ottoman Empire into new states after World War I. Lebanon, Syria, Jordan and Turkey itself feel threatened. But the extremist ideology has challenged all the Muslim world, and especially Saudi Arabia, Qatar and several other Gulf states that have either backed Islamist movements or quietly supported a more Islamist direction to Arab governments in the region.

IS has taken the Islamist movement to an extreme that few others want to follow. Countries such as Saudi Arabia, which over the years has encouraged Muslim movements around the world, find themselves directly threatened. They are now trying to stop all private funding of Islamist extremism while attempting to enlist global Islam in the fight against IS. Qatar, which strongly supported the Muslim Brotherhood in Egypt and elsewhere, is now attempting to distance itself from Islamism and its extreme offshoots.

But it is not only Muslims who feel threatened. The West is a prime target for IS, as it competes with al-Qaeda for leadership in global jihadism. The extremists see the US and its Western allies as the true enemies of Islam. And the threat of renewed terrorist action against the West is causing huge alarm. The fear is that the brutal ideology and extremist views of IS will spread to Muslim minorities in the West and radicalise an entire younger generation of alienated Muslim youth.

In fact, this is now already happening. One of the most alarming trends is the effectiveness of the online propaganda and the brutal IS videos of hostage beheadings. Far from repelling young Muslims in Europe, these appear to be attracting new recruits in large numbers. Thousands of youths, many of them well educated and from affluent middle-class homes, are travelling to Syria to join the IS rebels and take part in the battles across the region. A large number, estimated at well over 500, have come from Britain, and the British government as well has been horrified to find that several of the hostages beheaded on video were killed by a man identified as coming from the London area.

Why are so many Muslims in Europe attracted to a movement that seems determined to show itself more bloodthirsty, more ruthless and more barbaric than any other extremist group? And what can be done to stop the radicalisation of European Muslims and halt the advantage of extremist ideology?

Few governments have the answers. Britain has been looking at ways of countering the radicals for the past 10 years, ever since the bombing of the London Underground in 2005. The Government has set up liaison units to reach out to Muslim communities. It has supported an anti-extremist organisation called Quilliam, staffed largely by former Muslim militants who are now trying to counter extremism. It has proposed drastic new laws to curb travel to Syria and even suggested taking away the passports of those suspected of supporting IS.

So far, these measures have had little effect. In fact, many experts warn, they may even be counter-effective by increasing the sense of alienation among young Muslims and a fear among the majority of a growing Islamophobia that targets all Muslims as potential terrorists. For several factors affecting young people may be hard for any official body to counter. The first is the generation gap, especially in Muslim communities, where young people often feel pressurised by parents. In response, they take a defiant view and see the jihadist cause as a way of seeking personal glory and fame while also justifying their actions with religious motives.

The second factor is that many Muslim communities have fared poorly in Western societies – partly because of the difficulties of integration, partly because of poor economic backgrounds and partly because ill-informed preachers and Imams have focused on jihad and have been unable to offer practical advice to teenage Muslims growing up in Western societies with very different attitudes to marriage, relations between sexes and personal freedom.

Part of the appeal of IS is the fact that its brutality has been a main factor in its military success. The Iraqi Army fled in fear when it learnt of the mass executions that IS committed against other opponents. The panic of civilians in the face of IS fighters reinforces the image of IS as an all-powerful and all-conquering force. And this has given militant Islamists the belief that for the first time they are able to fight successfully against Arab regimes and their Western allies. This is a powerful recruiting message – especially when it is slickly put across online by people skilled in the use of social media and able to tailor the message to young Muslims in the West as well as in the wider Muslim world.

The West is therefore faced with a dilemma. If it intervenes in the fighting in Syria and Iraq, it will be again portrayed by Islamists as the enemy of Islam that  is seeking control of Arab lands and their oil wealth. If it fails to halt the IS advance, it will see many more young Muslims attracted to the movement. An IS victory may probably lead to the disintegration of Iraq, full-scale fighting between Sunni and Shia Muslims across the region and years of violence and civil war.

The Western compromise is to offer training, money, arms and air strikes though no actual ‘boots on the ground.’ But as every military expert has said, wars cannot be won by air strikes alone. The opponents of IS – mainly the Kurdish Peshmerga – are not strong enough to take on the militants who are well armed with captured weapons and flush with money from captured oil wells. Sooner or later, analysts say, America, Britain, France and other powers will have to offer troops to reinforce the Iraqi army.

The other dilemma is whether to compromise with Iran, the Assad regime and other groups that the West has strongly opposed. The old adage that ‘the enemy of my enemy is my friend’ does not work in the Middle East. Iran is happy to see IS weakened, but is still deeply suspicious of America. Assad wants to see IS powerful, because he knows that this will deflect Western attempts to remove him from power. Turkey is opposed to IS but has no wish to help Kurdish militants and is therefore happy to watch both sides inflict heavy losses on each other.

The situation is one of the most challenging the West has faced – both abroad and in accommodating Muslim communities at home. There are few clear answers. The Middle East will continue to take a heavy toll for years to come.