Updated! Updates are shown in quote text like this. Some scores are updated following app updates.

An Apps Experiment

Introduction

This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I’ve seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.

Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.

How I did it

I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.

I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.

I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @jordanlund@lemmy.world – Thanks, Jordan! This made for 20 apps that were tested.

Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @marvin@sffa.community, which was posted about a year ago in !meta@sffa.community (here).

I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.

Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.

In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.

Results

Out of a possible perfect 10, 7 apps displayed all markdown correctly:

Alexandrite - 10.0

Connect - 10.0

Jerboa (Official Android client) - 10.0

Photon - 10.0

Quiblr - 10.0

Summit - 10.0

Voyager - 10.0

Arctic - 9.3

Interstellar - 9.1

Lemmuy-UI - 9.0

Thunder - 8.9

Tesseract - 8.6

mlmym - 8.0

Racoon - 7.6

Boost - 7.3

Eternity - 7.0

Lemmios - 6.9

Sync - 6.9

Lemmynade - 6.1

Avelon - 5.7

More details of testing here

Disclaimers

Disclaimers

I Love Lemmy Apps (and their devs)

Lemmy apps devs work very hard, and invest a lot in the platform. Lemmy is better because they are doing the work that they do. Like, a LOT better. Everyone who uses the platform has to access it through one app or another. Apps are the face of the entire platform. Whether an app is a FOSS passion project, underwritten by a grant, or generating income through sales or ads, no one is getting rich by making their app. It is for the benefit of the community.

This is not meant to be a rating of the quality or functionality of any app. An app may have a high rating here but be missing other features that users want, or users may love an app that has a lower rating. This is just about how well apps handle markdown.

This is pretty unscientific

You’ll see my methodology above. I’m not a scientist. There is probably a much better way to do this, and I probably have biases in terms of how I went about it. I think it’s interesting and probably has some valuable information. If you think it’s interesting, let me know. If you think of a better way, PM me and I’d be happy to share what I have so you don’t have to start from scratch.

My only goal is to help the community

I do think that accurately displaying markdown should be a standard expectation of a finished app. I hope that devs use this as an opportunity to shore up the areas that are lagging, and that they have a set of standards to aim for.

I don’t have any Apple things

Sorry. This is just Android and Web review. If someone would like to see how iOS apps are doing, please reach out and I’ll share how we can work together to include them.

  • Plum@lemmy.world
    47·
    2 months ago

    Voyager fangirl here. I have used boost, sync, and jerboa, and voyager won pretty quickly.

    • 667@lemmy.radioEnglish
      24·
      2 months ago

      Having come from Apollo, the dev(s?) really focused on parity and it made the changeover simple.

      • Plum@lemmy.world
        6·
        2 months ago

        I used reddit is fun exclusively. Once I figured out compact post size, I was sold. It has every functionality and a great UI and it hardly ever barfs. And it has a Dracula theme for dark mode. Brilliant. Seamless.

        • fishpen0@lemmy.world
          3·
          2 months ago

          I think they mean natively detecting other Lemmy links and following them in-app instead of launching a browser. That’s usually what link-handling is shorthand for in the software industry

          I frequently find voyager opens a browser to a random other lemmy instead of sending me to the post inside voyager itself

  • idunnololz@lemmy.world
    31·
    2 months ago

    Hello! I’m the dev of Summit. Do you remember what Summit failed on? I would be very interested so I can fix it (seems like it failed maybe one or two things I’m guessing?)

    Anyways thanks for doing this!

  • gedaliyah@lemmy.worldOPM
    23·
    2 months ago

    Test post:

    Text (of 6)

    Is this italic? (1)

    Is this bold? (1)

    Is this strong? (1)

    Is this strikethrough? (1)

    Is this superscript? (1)

    Is this subscript? (1)

    Format (of 5)

    Quotes (1)

    Is this

    a blockquote?

    Is this separated?

    List (1)

    • Is
    • This
      1. A
      2. mixed
        • level
    • list?

    Code (1)

    def hello_world():
    print("Is this code block?")

    Is this inline code?

    Table (1)

    Is This a Table?
    Left? Center? Right?

    Horizontal line (1)

    Is there a line below?

    Spoilers (1)

    Is this expandable?

    Is this collapsible?

    Links (of 4)

    Is this a link? (1)

    Did it open in the app? (1)

    User: @gedaliyah@lemmy.world (Does it link to the user?) (1)

    Community: !lemmyapps@lemmy.world (Does it link to the community) (1)

    Images (of 3)

    Lemmy

    Is Lemmy above? (1)

    Lemmy

    Is Lemmy above? (1)

    Is Lemmy between the arrows? ➡️ Lemmy ⬅️ (1)

    • gedaliyah@lemmy.worldOPM
      7·
      2 months ago

      Detailed results:

      Earlier results

      My testing captures are below:

      Summit:

      Photon:

      Arctic:

      Interstellar:

      Lemmy-UI:

      Thunder:

      Tesseract:

      Quiblr:

      mlmym:

      Lemmios:

      Mlem:

      Boost:

      Eternity:

      Sync:

      Connect:

      Lemmynade:

      Avelon:

      • Aurelius@lemmy.world
        5·
        2 months ago

        Quiblr should now have each of the markdown criteria fixed. Thank you for the feedback and for your diligence in digging into the markdown and promoting a more consistent Lemmy experience across apps.

        Edit: Looks like I missed the “opening link in-app”. This should be updated now!

      • gedaliyah@lemmy.worldOPM
        1·
        2 months ago

        Arctic: Inline images fail, everything else works!

        Avelon: Strong and Subscript both fail. Code, Table, Spoilers, HR all fail. Everything else works!

        It looks like the code block and inline code are correct… Am I missing something?

        Lemmios: Subscript and Quotes fail, Code and Spoilers Fail

        Same thing, the code looks correct to me.

        Mlem: Subscript fails, Spoiler fails, 2nd inline image fails.

        • jordanlund@lemmy.world
          2·
          2 months ago

          The code block varies from app to app. I gave it a pass here because it did format it, it just didn’t format it in full color the way other apps do.

          • gedaliyah@lemmy.worldOPM
            2·
            2 months ago

            That is my experience as well. I’d like to eventually have some more complete review that covers more user requested features and I think having a quality code display would be one of them. There is a lot of discussion about and sharing code, so for some communities it would make a huge difference.

  • TrickDacy@lemmy.world
    7·
    2 months ago

    Surprised to see Boost that low in the rankings. Literally the only issue I’m aware of is the spoiler syntax isn’t supported. I’ve always considered it much more solid in the way it feels compared to the other apps. I think I’ve used most of them.

    • gedaliyah@lemmy.worldOPM
      8·
      2 months ago

      I was not joking about this not being an indication of overall quality. My favorite apps are lower on this list but have other great features! I hope to have a better resource in the future to provide App reviews for different features.

      Spoilers are pretty important though. That’s one or like to see in every app.

      • TrickDacy@lemmy.world
        1·
        2 months ago

        Gotcha. Like some other users, I’m quite curious to see the details of how you arrived at these scores. The spoiler thing can be important, but for some reason it doesn’t bother me personally all that much. I’m very curious if there are other rendering issues with Boost outside of that one.

  • jordanlund@lemmy.world
    7·
    2 months ago

    @gedaliyah@lemmy.world iOS testing, not sure how you score these so I just listed out the broken stuff.

    Arctic - Link opens in App. Headings fail, images fail, everything else looks fine.

    Avelon - Link opens in browser, not app. Manually went to test post. Bold+Italic fails (Italic works, not Bold). Table fails. Horizontal Rule fails. Spoiler fails. Everything else looks good.

    Bean - Last updated 7 months ago, comments on the app say it’s abandoned. Link opens in browser, not app. Manually went to test post. Text formatting block fails so hard, it’s not even visible(!) Heading fails. Code Block fails, Inline Code fails. Links and Image work, but not inline, only at the bottom of the post. Table fails. Horizontal rule fails.

    CheeseBot - Did not test. $2.99, no free version.

    Lemmios - Link opens in app. Everything looks and works great EXCEPT Spoilers.

    Mlem - Link opens in browser, not app. Manually went to test post. As with Lemmios, everything looks and works great EXCEPT spoilers.

    Remmel - Instant fail. No development in 2 years, unable to even add an instance or an account. Non-starter.

    Thunder - Hard to test. Lots of lag for some reason. Link opens in browser, not app. Manually went to test post. That being said, EVERYTHING worked. The lag may have been because I had just linked my account. Testing everything above, then coming back to Thunder, I found it fast and responsive.

    Voyager - Link opens in app. EVERYTHING worked. No notes.

    So, ranking them:

    Voyager - EVERYTHING worked. No notes.

    Thunder - Everything worked, but laggy to start with when using a year old account with lots of data. Once it caught up, everything was fine. Would probably be great with a new account.

    Lemmios - Link opens in app by default. Spoilers don’t work.

    Mlem - Link opens in browser by default but is user configurable. Spoilers don’t work.

    Arctic - A few minor failures.

    Avelon - A few more failures than Arctic.

    Bean - Hey, it works better than Remmel. Probably abandoned.

    Remmel - Instant fail.

    CheeseBot - Did not test. $2.99, no free version.

  • scottywh@lemmy.world
    4·
    2 months ago

    Your results with Sync are drastically different than mine.

    I’m seeing 6 out of 6 and 5 out of 5 on the first two tests.

    • gedaliyah@lemmy.worldOPM
      8·
      2 months ago

      From the image above, it looks like the subscript did not render (and the superscript did not terminate correctly, although that would not be a deduction). The two separate quotes were rendered as one continuous block. This image doesn’t show the table, but on my device the table does not render the correct column alignment. Maybe you can show me what you mean?

      • scottywh@lemmy.world
        4·
        2 months ago

        The table looks to be correct to me… Looks like maybe you’re right about the sub and superscripts actually though.

        • KubeRoot@discuss.tchncs.deEnglish
          7·
          2 months ago

          In the table, the Center and Right text is supposed to be center and right-aligned within their respective cells

        • DieguiTux8623@feddit.it
          6·
          2 months ago

          Markdown rendering libraries do their best, and Raccoon uses one of the best in the KMP ecosystem.

          It would be nice to know what the benchmarks were to see whether anything can be done to improve the results, in any case I feel sorry for this low rating and this is why I advise people to switch to another app if this is not acceptable.

          BTW The Raccoon project never got real “traction” and I have other side projects I would like to do in my spare time so it is what it is unfortunately.

          • gedaliyah@lemmy.worldOPM
            2·
            2 months ago

            SO, in reviewing the scores, I realized that Raccoon does not render tables correctly, but also, I was checking some formatting that was within a table, which was also not rendered correctly. This resulted in the app getting marked down twice when it shouldn’t have. I am posting an update shortly that will correct this.

            I am very sorry for this oversight. I should have gone about this part differently.

            • DieguiTux8623@feddit.itEnglish
              1·
              2 months ago

              Could you please share the score for each point and make it clear the mechanism through which scores have been assigned? Raccoon has indeed great flaws in how it renders Markdown and the maintainers are well aware of them, but for transparency it would be better to disclose all the details.

              • gedaliyah@lemmy.worldOPM
                1·
                2 months ago

                Absolutely - I should have included this information from the start and I am sorry that I did not. I hoped to avoid information overload, but clearly it is a lot more helpful to the community to see the specifics.

                See the detailed results here

    • gedaliyah@lemmy.worldOPM
      1·
      2 months ago

      Unfortunately, there was a flaw in the original test that essentially penalized the app twice when it shouldn’t have. I have corrected this and the updated score is above.

  • IMALlama@lemmy.world
    3·
    2 months ago

    Jerboa user checking in. My biggest gripes with it are:

    • not always showing me that I have things in my inbox when refreshing my feed
    • links from a post to another post seem to render in my default browser (Firefox) vs Jerboa
    • every once in a while gbord’s spell check stops working. I haven’t experienced this in other apps, so… maybe something is going on here?

    Everything else is pretty solid though. The only nice to have feature would be saving draft replies locally automatically. Reddit is fun did that and spoiled me. Nothing is worse than starting a long reply, life getting in the way, coming back to Lemmy, and being greeted b my refreshed feed.

    Your other post on this same topic, with a cross post to this one, renders this way when I clicked the cross post link: