Lemmy Development Update 2023-09-22

Some years ago we used to post weekly development updates to let the community know what we are working on. For some reason we stopped posting these updates, but now we want to continue giving you information every two weeks about the recent development progress. This should allow average users to keep up with development, without reading Github comments or knowing how to program.

We’ve been working towards a v0.19.0 release of Lemmy, which will include several breaking API changes. Once this is ready, we’ll post the these changes in dev spaces, and give app developers several weeks to support the new changes.

This week @nutomic finished implementing the block instance feature for users. It allows users to block entire instances, so that all communities from those instances will be hidden on the frontpage. Posts or comments from users of blocked instances in other communities are unaffected. He also reworked the 2-Factor-Authentication implementation, with a two-step process to enable 2FA which prevents locking yourself out. Additionally he is reworking the API authentication to be more ergonomic by using headers and cookies. Finally he is adding a feature for users to import/export community follows, bocklists and profile settings.

@dessalines is currently implementing a redesign of the join-lemmy.org website. He is also keeping the lemmy-js-client updated with the latest backend changes 1 2 3.

@phiresky optimized the way pagination is implemented. He is also fixing problems with federation workers which are causing test failures and performance problems in the development branch. These problems were introduced during a complex rewrite of the federation queue which was recently finished, and is thought to allow Lemmy federation to scale to the size of Reddit.

@SleeplessOne1917 is implementing remote follow functionality, which makes it easy to follow communities from your home instance while browsing other instances. He is also fixing problems with the way deleted and removed comments are handled .

@codyro and @ticoombs have been making improvements to lemmy-ansible, including externalizing the pict-rs configuration, adding support for AlmaLinux/RHEL, cleaning up the configuration, as well as versioning the deploys. These will make deploying and installing Lemmy much easier.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

Cheradenine,

Thank you, all of that sounds like very good news.

Thanks to all the people working on this.

SheeEttin,

If these are API-breaking changes, shouldn’t you bump the major version? semver.org

maegul,
@maegul@lemmy.ml avatar

I think you’re forgetting this:

  1. Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.
SheeEttin,

Fair enough.

From an end user perspective, it feels like it’s operationally stable, though I don’t know about developmentally stable. Maybe it’s worth a 1.0 release soon. Lots of people are running it in production now.

ares35,
@ares35@kbin.social avatar

see also: perpetual beta

maegul,
@maegul@lemmy.ml avatar

Yea. I feel like once it can scale pretty well and has been for a bit of time that would be a good opportunity to release 1.0. But another major factor here is that the backing and sustainability of the project is still up in the air, so the flexibility of breaking changes is maybe rather valuable for a while.

nutomic,
@nutomic@lemmy.ml avatar

Before 1.0 we definitely need to do an API cleanup, the paths are a real mess. However that will require lots of breaking changes so Im not sure when we can do it.

poVoq,
@poVoq@slrpnk.net avatar

0ver.org

maegul,
@maegul@lemmy.ml avatar

Ha. Well yea, devs need to learn to commit at some point I guess.

dessalines,

I feel their pain lol, which is why I’m also super-hesitant to pull the trigger on 1.0.0 for lemmy.

We’re still hobby software created mostly by a few devs, yet people expect us to have the same stability and resources as multi-million-dollar corporations with hundreds of employees.

maegul,
@maegul@lemmy.ml avatar

We’re still hobby software created mostly by a few devs, yet people expect us to have the same stability and resources as multi-million-dollar corporations with hundreds of employees.

Oh for sure … for me, you go ahead and break backwards compatibility all you need to. Though there might be a weird phase coming up where a number of people are using apps whose development has slowed or stalled and so won’t be able to get updated.

Otherwise, my comment about “committing” was targeted at some of the notable zeroVer projects: react native, threejs, hugo and neoVim.

dessalines, (edited )

We’re reserving 1.0.0 for a mostly unchanging and stable API. That def isn’t the case currently, as we’ve been rapidly adding features, changing api objects, etc. So minor versions (and usually not patch unless it’s security related), signify breaking api or config changes.

IzzyData,
@IzzyData@lemmy.ml avatar

Great work. Thank you for the update Nutomic.

mechatux,

deleted_by_author

    •
    poVoq,
    @poVoq@slrpnk.net avatar

    Both is kind of useful, but making the distinction UI wise is not an easy task.

    Die4Ever, (edited )
    @Die4Ever@programming.dev avatar

    I guess

    “Block communities of abc.xyz”

    vs

    “Block users of abc.xyz”

    I also think both options would be useful

    MBM,

    Similar options for defederating would also be nice, like “disallow users from this instance from posting/commenting in your communities” vs “do not fetch communities from this instance”. There may be some higher-priority things though.

    Cheradenine,

    Exactly that, if i block an instance, i want it blocked.

    Zeth0s,

    I would say that that’s is what you want. We don’t have a survey on user intentions.

    I for instance would like to block furry instances, as an example, but I have nothing against their users. I don’t care if someone is into furry, I simply don’t want to see it myself

    Coelacanth,
    @Coelacanth@feddit.nu avatar

    Completely agree. I can think of many examples of an instance whose content I don’t want on my frontpage (foreign language instances for example) but whose users I still want to interact with in communities on other instances.

    notabot,

    Just adding another voice agreeing that I just want to block instance posts, not comments from their users.

    If people really don’t want to see anything from an instance then they should join an instance that is bit federated with the one in question.

    If blocking all users from an instance does get implemented I believe it should be separate to blocking posts as they’re very different use cases.

    WarmSoda,

    You can join an instance that has defederated and it’ll be exactly what you’re looking for.

    Rentlar,

    I agree with this. Instance blocking should be for that instance’s content. If you are opposed to that instance’s users’ ideology then pick servers that match what you want.

    anonymoose,
    @anonymoose@lemmy.ca avatar

    Strongly disagree. I want to block certain instances from my feed to remove posts I’m not interested in (eg NSFW, repost bots), but I have nothing against the users.

    If users are blocked too, please add that behind a setting toggle.

    ladfrombrad,
    @ladfrombrad@lemdro.id avatar

    but I have nothing against the users.

    Absolutely, and in 12+ years of using reddit the only accounts funnily enough I’ve had to block are admin accounts, and I liken the above to users from other subreddits being able to able to interact in other subs until they either

    • get banned by the sub /c/ mods
    • individual users using the block function

    Tarring users from instances is like using that SaferBot bullshit which I’ve never liked.

    anonymoose,
    @anonymoose@lemmy.ca avatar

    Funnily enough, I block very liberally.

    I find that having the option to block by keywords, instances, communities and users let’s me curate my feed exactly as I want. What I wouldn’t want is for the blocks to be too broad of a brush (like in the users of instances example), which would lead to missing out on valuable conversations.

    ladfrombrad,
    @ladfrombrad@lemdro.id avatar

    Exactly, and the way it work(ed) on reddit was a non-plus from the start, especially if you modded moderately large subs hence me never blocking anyone other than those annoying accounts.

    The art of ignoring is key ;)

    kilgore_trout,

    To me what you are asking for makes no sense.

    lemann,

    Thank you for the update 😁

    Also devs please feel free to take a well-deserved break whenever you feel like - wouldn’t want to see anyone getting burned out from spending the majority of available time working on Lemmy. I can imagine the reddit migration has shaken a few things out of order especially

    CannotSleep420,
    @CannotSleep420@lemmygrad.ml avatar

    I contribute to lemmy in part because I would lose my mind if the only software I wrote that people actually use was crapware for my corpo job.

    nutomic,
    @nutomic@lemmy.ml avatar

    Thanks for the concern. Personally I took plenty of time off during summer. Now Im motivated to code again, and honestly I would get really bored if I did nothing.

    Blaze,
    @Blaze@discuss.tchncs.de avatar

    Great news, thanks for the update!

    Illecors,

    @phiresky optimized the way pagination is implemented. He is also fixing problems with federation workers which are causing test failures and performance problems in the development branch. These problems were introduced during a complex rewrite of the federation queue which was recently finished, and is thought to allow Lemmy federation to scale to the size of Reddit.

    I can’t wait for this to get into action! Separating the federation into a separate process with the ability of using a separate physical resource is so good!

    Thank you!

    Rentlar,

    Thanks for the update! I think it’s a good idea to get the word out there more often on what’s being worked on and how progress is going at a regular interval.

    ulu_mulu,

    Amazing job as always!

    Oha,
    @Oha@lemmy.ohaa.xyz avatar

    Thank you for your work. I love seeing that my money actually goes to something great

    mojo,

    Looking at the Liberapay, it’s shocking Lemmy only gets $332 a month to fund this entire social media. Compare that to the totally “grass roots” normal NFT growth where they immediately had millions poured into them from venture capitalist. As it turns out, actual grass roots social media without profit incentive isn’t profitable! And that’s how you know we’re on the right path.

    dessalines,

    For as many users as lemmy has now, its kind of astonishing how little donations we have, like less than the average youtuber / streamer with a patreon. Its more when we sum up the other platforms, but I’d really like us to be able to add more full-time devs and grow the coop.

    And not just us of course, but open source software in general needs so much more funding than its currently getting. If you use open source software, consider donating to those devs!

    Once I finish the join-lemmy.org site redesign, I’ll put a section on the donation page that sums all these up for transparency’s sake, and we’ll probably try to have a once-per-year donation push to try to make sure we get fully funded.

    Corkyskog,

    I wonder how many people don’t understand Lemmy and thought when they were donating to their instances it benefitted lemmy development as a whole? I see many posts about donating to your instance, but little to donating to devs. Do any instances share their donations?

    Gust3r,
    @Gust3r@mastodon.world avatar

    @Corkyskog @dessalines also to third party apps...

    dessalines,

    Absolutely, especially the open source ones. We should be linking their donation pages wherever we can.

    dessalines,

    Instance runners should also get donations, if not just for the hosting costs (which shouldn’t be too much), then for their labor time spent moderating and building spaces.

    I’m sure many of them also do contribute to lemmy’s dev upstream.

    kolektyw_szmer,
    @kolektyw_szmer@szmer.info avatar

    A shared donation page might encourage them to promote it. One where there would be all lemmy dev funding options, but admins could also add theirs. Sounds like a low hanging fruit.

    We’ll also discuss announcing a cut of our donations going upstream in our collective.

    dessalines,

    For sure! I’ll add it to my TODOs for the new joinlemmy site.

    kolektyw_szmer,
    @kolektyw_szmer@szmer.info avatar

    It could be a page on the instance with half based on a form to be filled out in the admin pages, and the other half with Lemmy funding.

    silent_water,
    @silent_water@hexbear.net avatar

    youtubers/streamers have a parasocial thing going with their audience that makes the idea of donating a smaller mental step for their audience (senpai might notice me if I donate type brainworms). FOSS projects historically have really struggled with funding, unless they’re able to secure funding from an org/corporation.

    wiki_me, (edited )

    For as many users as lemmy has now, its kind of astonishing how little donations we have, like less than the average youtuber / streamer with a patreon. Its more when we sum up the other platforms, but I’d really like us to be able to add more full-time devs and grow the coop.

    Why look at youtubers when you can look at a open source project?, look at misskey which is very similar , it makes about 4k while having about 10k monthly active users, that’s about 0.4 dollar per user.

    Lemmy has about 40k monthly active users and makes about 3962 , about 0.1 dollar per user.

    If you will push the conversation rate to be as high as misskey, that should give you currently about 16K a month (40k * 0.4).

    I have a few ideas about how to increase it, i can open a issue throwing some ideas, for starter (I don’t remember if i said this before) the part in the UI where people are suppose to learn lemmy wants donations (the little heart), is probably very hard to notice.

    fbmac,
    @fbmac@lemmy.fbmac.net avatar

    Maybe opt-in ads? It could be opt-in for both the instance and the end user

    I’m not demanding it to be opt-in, just taking the assumption that you don’t want to just put ads there, and that there would be a backslash if you just put them there

    Or some kind of “lemmy gold”

    hillbicks,
    @hillbicks@feddit.de avatar

    I regularly comments from users who were not aware or the financial situation. Maybe “we” need to promote it a bit more. But it is 332 per week, not month. At the beginning of the last migration it stood at 40 something, so we at least got some traction.

    Thanks for the update and thanks for the work guys, I really appreciate it

    fmstrat,

    We’ve been working towards a v0.19.0 release of Lemmy, which will include several breaking API changes. Once this is ready, we’ll post the these changes in dev spaces, and give app developers several weeks to support the new changes.

    Thank you for the update and the heads up.

    How will this be announced, and what specifically does several weeks mean? Since Lemmy goes beyond Mobile Apps to all kinds of systems including moderation tools, auto-purgers, bots, CSAM, auto-subscribers, searchers, etc, breaking changes to the API can have far-reaching impacts.

    Could something be set up specifically for breaking-change announcements where participants could be alerted? Even just a Breaking Changes issue that could be followed would work nicely.

    Thank you again.

    nutomic,
    @nutomic@lemmy.ml avatar

    When we are ready to publish the first release candidate, we will make a post that lists all the breaking changes. You can follow !announcements via rss reader to get notified about it. We will also share it in different Matrix chats, and I’m sure it will get upvoted to the frontpage as well.

    xusontha,

    Do you have any sort of timeline for when 0.19 or the release candidates will become available? I only wonder because I’m eager to check out some of the new features that have been mentioned here and on Github

    nutomic,
    @nutomic@lemmy.ml avatar

    See here: lemmy.ml/post/5711722

    fmstrat, (edited )

    For the truly breaking changes like API auth, pagination, and TOTP, is there a reason you don’t roll the deprecation like most software?

    I.E. 0.19 supports both methods, and 0.20 deprecates the old one? This way developers aren’t caught off guard if they’re not following (which will get worse as time goes on), and allows development using official releases vs RCs.

    For instance, if I want to update my app now, I have to release it with an RC library. If there was a version between deprecation, I could update at any point during the official 0.19 lifespan.

    nutomic,
    @nutomic@lemmy.ml avatar

    In case of pagination both old and new variants are supported in 0.19 (see my reply to fmstrat above). TOTP is currently broken so it wouldnt make sense to keep supporting the old version. In case of auth it would be possible to keep backwards compat, but keep in mind that we are only two fulltime devs with tons of other things to work on. If we spend a lot of time on this, it means less time for other important tasks. Besides you can support both 0.19 auth and 0.20 auth at the same time by sending auth as param and header/cookie.

    fmstrat,

    Actually, as I start to do updates, it looks like the JS library still has page along with page_cursor (and the auth can be run in parallel) which seems to be great news! Does this mean I can continue to use page for pagination until .19 officially releases and rolls out, and then switch over?

    nutomic,
    @nutomic@lemmy.ml avatar

    0.19 supports both page and page_cursor. We will probably get rid of page in 0.20.

    klemptor,

    This week @nutomic finished implementing the block instance feature for users.

    Thank youuuuuu!

    mouth_brood,

    Donated! Keep up the good work

    cetra3,
    @cetra3@lemmy.ml avatar

    Excellent work! Keep it up!

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • uselessserver093
  • Food
  • aaaaaaacccccccce
  • [email protected]
  • test
  • CafeMeta
  • testmag
  • MUD
  • RhythmGameZone
  • RSS
  • dabs
  • Socialism
  • KbinCafe
  • TheResearchGuardian
  • Ask_kbincafe
  • oklahoma
  • feritale
  • SuperSentai
  • KamenRider
  • All magazines
    •