Salaam and welcome,
Allah and His Messenger only left ONE book for us. If an explanation of the book was extremely important then the Messenger would have dictated it, compiled it, verified it and left it for us. He knew well what happens to oral traditions over time. When Allah proclaimed completion of religion, no other book other than the Qur'an existed.
I feel religion has to be applied with wisdom otherwise it results in chaos. Allah exhorts us to use our intellect when it comes to deen. As your research shows most of Muslim inventors were rationalists!!
Sure we can read hadees as an attempt to record history but must only take those reports seriously which match with the Qur'an. I just finished watching a YOUTUBE discussion by a panel on the issue of three TalaqS (divorce) hosted by Geo's Alif programme. All the panelists agreed that there was no room for instant three talaaqs per the Qur'an but then some admitted there could be a possibility of acceptance because Umar the caliph accepted a case presented to him (with reservations), and then such and such compation also believed in it, etc. In other words, they found it hard to ignore "historical" reports and did not even think that how can Umer accept an action against the Qur'an?? Would have been easier to dismiss it it as a fabrication, but they bend over backwards to understand what Umer did!!! The Shia scholar would have none of it and stated that their jurispudence dont accept such rationalisations as for them, when no one has the authority to change what Allah and His messenger decreed. Not even the companions!
All I know is that what ever is being fed by traditional scholars to the sick Muslim Ummah as medicine is not working. No need for proof as we all know what the facts are. It is about time the patient sits up and ask the doctor what is going on!! It has been several hundred years of our downfall and it is about time we start asking questions and raise important issues. We need to look for a better medicine and seek new solutions. Only blind followers keep falling into the ditch with others. We need to make some serious choices and this what this forum is about. We will differ with grace but not stifle civilised discussion.