The problem that conversations go off-topic is as old as the hills. A 'branch-off' structure was possible as soon as mailing lists and bulletin boards were invented. A forum like this is a deliberate decision against that style.
Contributors are trusted with the maturity to link-out of an unrelated conversation and start a new topic. That means that if everyone posts intelligently, you should find under any given heading only information that is actually relevant to that topic - and if an interesting new but only vaguely related point emerges you can find it under the appropriate description, not under some unrelated heading.
While preferences differ and there are points for and against, (https://blog.codingh...flat-by-design/ ; http://www.fullcirc....eadedlinear.htm )
There are fancy alternatives [http://www.drewgl.com] and when our friends at "SingInst" (as it was then) embraced the 'lesswrong' format we thought very seriously about these. But ultimately, technical options in themselves do not necessarily enhance discussion quality. It only works if users are committed to it. Our community is small and our resources are limited. We have 'maxed' the forum platform to incorporate enhanced features including subscriptions, signalling cookies, personalised notifications, linklists, libraries, regimens, quizzes, self-moderated groups, tags, keywords etc. and even these extra features are barely used. LongeCity's mission is not to pioneer new ways of curating internet discussions.
To belabour the point in a way: I have -hopefully correctly- identified the above as the OP's 'main point'. There are other -to me, unrelated- issues thrown in, including:
1- while you can search the forum easily you can't search individual threads. - Yes that is a deficit that we could look into given resources. But unless the thread is super-long, you can search it using the browser search function.
2- The rating system - this really is a separate issue, which has been hotly discussed. Again, the decision to go away from "up/down" votes was very deliberate.
3- "the most popular conversation is always at the top at any given time" - that is the case here as well, but certainly not always a good thing.
Forums in the LongeCity style once used to be more ubiquitous for various reasons. First, wikipedia emerged to allow for more orderly pooling of crowdsourced knowledge.
Then social media took over 95% of the remaining space - it is a wonder that forums still exist at all.
Those dominate that have a big userbase (reddit, chans, craigslist...) based on their near limitless range of topics.
In contrast, LongeCity is positioned with a clear mission and -although developed in many different aspects and considerations- theme for discussion.
Also popular still are straightforward "One Question - answered" forums and "One Article - comments" forums. Most discussions on LongeCity are not in that pattern.
For us, the 'old school' forum structure still works better than any other system that we would be able to implement. I say this because that question is not 'closed' but under active, if occasional review.
So: if you have any tangible models in mind that we could seamlessly migrate to which would work much better for our mission, please let us know.