There are other narrations along the same line which depict a merciful era of Kingship after caliphate:
حَدَّثَنَا أَحْمَدُ بْنُ النَّضْرِ الْعَسْكَرِيُّ، ثنا سَعِيدُ بْنُ حَفْصٍ النُّفَيْلِيُّ، ثنا مُوسَى بْنُ أَعْيَنَ، عَنِ ابْنِ شِهَابٍ، عَنْ فِطْرِ بْنِ خَلِيفَةَ، عَنْ مُجَاهِدٍ، عَنِ ابْنِ عَبَّاسٍ، قَالَ: قَالَ رَسُولُ اللهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ: أَوَّلُ هَذَا الْأَمْرِ نُبُوَّةٌ وَرَحْمَةٌ، ثُمَّ يَكُونُ خِلَافَةً وَرَحْمَةً، ثُمَّ يَكُونُ مُلْكًا وَرَحْمَةً، ثُمَّ يَكُونُ إِمَارَةً وَرَحْمَةً، ثُمَّ يَتَكادَمُونَ عَلَيْهِ تَكادُمَ الْحُمُرِ فَعَلَيْكُمْ بِالْجِهَادِ، وَإِنَّ أَفْضَلَ جهادِكُمُ الرِّبَاطُ، وَإِنَّ أَفْضَلَ رباطِكُمْ عَسْقَلَانُ
Mujahid reported from Abdullah Ibn-e-Abbas that Prophet Muhammad (sws) said “The beginning of this matter is with Prophethood of Mercy, then it will be caliphate of mercy, then it will be kingship of mercy, then it will be rulership of mercy, then they will fight over it like donkeys. So you must do Jihad and the best Jihad is Ribat and the best Ribat is that of Ashkelon”.
Narrated in:
Moajjam ul Kabir Tabrani-11/88 (Number 11138)
– Classed Sahih by haithmi
(Source: Majma al-Zawaid by Haithmi-5/189 (8964))
– Classed Sahih by Shiekh Albani
(Source: Silsila Ahadith e Sahiha by Shiekh Nasir ud Din Albani-7/802 (Number 3270))
On the basis of this narration, Ibn-e-Taimiya wrote:
فكانت نبوّةُ النبي – صَلَّى اللَّهُ عَلَيْهِ وَسَلَّمَ – نبوّة ورحمة، وكانت خلافةُ الخلفاء الراشدين خلافةَ نبوّةٍ ورحمة، وكانت إمارةُ معاوية مُلكًا ورحمة، وبعده وقع مُلكٌ عَضُوض.
“The prophethood of the Prophet – may God’s prayers and peace be upon him – was prophethood of mercy, and the succession of the Rightly-Guided Caliphs was a caliphate on footsteps of Prophethood of mercy, and the Emirate of Muawiyah was kingship of mercy, and after him came a bitter king”.
(Source: Jamay al-Masail by Ibn-e-Taimiya-151)
In another variant:
حَدَّثَنَا مُحَمَّدُ بْنُ جَعْفَرِ بْنِ أَعْيَنَ، ثَنَا أَبُو بَكْرِ بْنِ أَبِي شَيْبَةَ، ثَنَا زَيْدُ بْنُ الْحُبَابِ، ثَنَا الْعَلَاءُ بْنُ الْمِنْهَالِ الْغَنَوِيُّ، حَدَّثَنِي مُهَنَّدُ الْقَيْسِيُّ، وَكَانَ ثِقَةً، عَنْ قَيْسِ بْنِ مُسْلِمٍ، عَنْ طَارِقِ بْنِ شِهَابٍ، عَنْ حُذَيْفَةَ بْنِ الْيَمَانَ قَالَ: قَالَ رَسُولُ اللَّهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ إِنَّكُمْ فِي نُبُوَّةٍ وَرَحْمَةٍ، وَسَتَكُونُ خِلَافَةٌ وَرَحْمَةٌ، ثُمَّ يَكُونُ كَذَا وَكَذَا، ثُمَّ يَكُونُ مُلْكًا عَضُوضًا، يَشْرَبُونَ الْخُمُورَ، وَيَلْبِسُونَ الْحَرِيرَ، وَفِي ذَلِكَ يُنْصَرُونَ إِلَى أَنْ تَقُومَ السَّاعَةُ
لَمْ يَرْوِ هَذَا الْحَدِيثَ عَنِ الْعَلَاءِ بْنِ الْمِنْهَالِ إِلَّا زَيْدُ بْنُ الْحُبَابِ “
Tariq bin Shahab reported from Huzaifa bin Yaman that Prophet Muhammad (sws) said “You are in the era of prophethood of mercy, then there will be caliphate of mercy, then there will be such-and-such, then there will be a stinging kingship, they will drink alcohol, wear silk, and in that they will be helped until the final hour comes”.
Tabrani said: “No one narrated this hadith on the authority of Al-Ala bin Al-Minhal except for Zaid bin Al-Habbab”.
Narrated in:
Moajjam ul Awsat Tabrani-6/345 (Number 6581)
– Classed Sahih by haithmi
(Source: Majma al-Zawaid by Haithmi-5/189 (8960))
__
As for the narration of Musnad Ahmed asked in the question:
حَدَّثَنَا سُلَيْمَانُ بْنُ دَاوُدَ الطَّيَالِسِيُّ، حَدَّثَنِي دَاوُدُ بْنُ إِبْرَاهِيمَ الْوَاسِطِيُّ، حَدَّثَنِي حَبِيبُ بْنُ سَالِمٍ، عَنِ النُّعْمَانِ بْنِ بَشِيرٍ، قَالَ: كُنَّا قُعُودًا فِي الْمَسْجِدِ مَعَ رَسُولِ اللهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ، وَكَانَ بَشِيرٌ رَجُلًا يَكُفُّ حَدِيثَهُ، فَجَاءَ أَبُو ثَعْلَبَةَ الْخُشَنِيُّ، فَقَالَ: يَا بَشِيرُ بْنَ سَعْدٍ أَتَحْفَظُ حَدِيثَ رَسُولِ اللهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ، فِي الْأُمَرَاءِ؟ فَقَالَ حُذَيْفَةُ: أَنَا أَحْفَظُ خُطْبَتَهُ، فَجَلَسَ أَبُو ثَعْلَبَةَ، فَقَالَ حُذَيْفَةُ: قَالَ رَسُولُ اللهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ: ” تَكُونُ النُّبُوَّةُ فِيكُمْ مَا شَاءَ اللهُ أَنْ تَكُونَ، ثُمَّ يَرْفَعُهَا إِذَا شَاءَ أَنْ يَرْفَعَهَا، ثُمَّ تَكُونُ خِلَافَةٌ عَلَى مِنْهَاجِ النُّبُوَّةِ، فَتَكُونُ مَا شَاءَ اللهُ أَنْ تَكُونَ، ثُمَّ يَرْفَعُهَا إِذَا شَاءَ اللهُ أَنْ يَرْفَعَهَا، ثُمَّ تَكُونُ مُلْكًا عَاضًّا، فَيَكُونُ مَا شَاءَ اللهُ أَنْ يَكُونَ، ثُمَّ يَرْفَعُهَا إِذَا شَاءَ أَنْ يَرْفَعَهَا، ثُمَّ تَكُونُ مُلْكًا جَبْرِيَّةً، فَتَكُونُ مَا شَاءَ اللهُ أَنْ تَكُونَ، ثُمَّ يَرْفَعُهَا إِذَا شَاءَ أَنْ يَرْفَعَهَا، ثُمَّ تَكُونُ خِلَافَةً عَلَى مِنْهَاجِ نُبُوَّةٍ ” ثُمَّ سَكَتَ،
Habib bin Salim reports from Noman bin Basheer that he said that we were sitting in the mosque. Basheer stopped what he was saying. In the meantime, Abu Thaalba Khoshni, may God be pleased with him, came and said: Bashir bin Saad! Do you remember any prophetic hadith about rulers? Huzaifah said: (In this matter) I remember His (sws) sermon. Abu Thaalbah sat down and Huzaifah said: The Messenger of Allah, may God bless him and grant him peace, said: Prophethood will continue for some time according to the will of Allah the Exalted, then He will remove it when He wills. After the prophethood, there will be a caliphate in the footsteps of Prophethood for some time according to the will of Allah, then Allah will abolish it, then according to the decision of Allah there will be a kingdom for some time in which there will be oppression and abuse. Finally, it will also end, then there will be a forced monarchy, it will decline after some time. After that there will be a caliphate again in the footsteps of Prophethood then He (sws) became silent.
قَالَ حَبِيبٌ: ” فَلَمَّا قَامَ عُمَرُ بْنُ عَبْدِ الْعَزِيزِ، وَكَانَ يَزِيدُ بْنُ النُّعْمَانِ بْنِ بَشِيرٍ فِي صَحَابَتِهِ، فَكَتَبْتُ إِلَيْهِ بِهَذَا الْحَدِيثِ أُذَكِّرُهُ إِيَّاهُ، فَقُلْتُ لَهُ: إِنِّي أَرْجُو أَنْ يَكُونَ أَمِيرُ الْمُؤْمِنِينَ، يَعْنِي عُمَرَ، بَعْدَ الْمُلْكِ الْعَاضِّ وَالْجَبْرِيَّةِ، فَأُدْخِلَ كِتَابِي عَلَى عُمَرَ بْنِ عَبْدِ الْعَزِيزِ فَسُرَّ بِهِ وَأَعْجَبَهُ “
Habib Said: “When Omar bin Abdul Aziz rose to power and Yazid bin Noman bin Basheer was among his companions, so I wrote to him with this hadith, reminding him of it. I said to him: I hope that the Commander of the Faithful – meaning Omar – is after the biting kingdom and tyranny, when that letter reached Omar bin Abdul Aziz, He liked the interpretation and praised it”.
Narrated in:
Musnad Ahmad-30/355 (Number 18406)
Musnad Al Buzzar-7/223 (Number 2796)
Musnad Abi Dawud Tayalsi-1/349 (Number 439)
– Classed Sahih by Haithmi
(Source: Majma al-Zawaid by Haithmi-5/188 (8960))
– Classed Sahih by Shiekh Albani
(Source: Silsila Ahadith e Sahiha by Shiekh Nasir ud Din Albani-1/34 (Number 5))
– Classed Hassan by Shiekh Shuaib al-Arnaut.
There are many narrations from Huzaifah (rta) regarding the ups and downs in Muslims after demise of Prophet Muhammad (sws), it is not possible to mention all of these here. Following work in Hadith Project sheds further light on other narrations in this regard:
نبی صلی اللہ علیہ وسلم کے بعد خیر و شر