Yes, though that’s true of other methods.
Another big factor is that its very inconvenient to buffer vs tanks on either end, for transmission breaks that take time to repair, uneven energy supply/demand and stuff like that. Or even just capacitance.
A big old tank of oil on either end is cheap.
I’m not trying to shill for hydrogen or anything (I don’t like hydrogen), but this is definitely an issue.
I am not a power engineer, but I do know the capital costs for the wire and components all along the way is massive. They’re complicated, and they require a lot of expensive (and probably carbon intensive) materials.
Basic physics dictates it. Its more complicated than small scale DC/AC current with negligible transmission time you’re likely thinking of.
Maintenance is a pain, too. HV wires (especially the crazy DC ones) are extremely, extremely dangerous and basically can’t be near anything.
I’m not sure about installation labor costs vs a pipeline though.
Power transmission is hard.
The wire, the complex components, keeping it all in phase and steady and not exploding, the maintenance…
I cannot emphasize this enough. People tend to trivialize this when talking about remote production, but moving electricity long-distance is basically the hardest part. And pipes really are dead-simple in comparison.
It’s also why local production is so appealing.
…Regardless of the code assistant, I’m not seeing any cause for alarm in my personal usage, yet. Feature requests and issues are getting closed, even urgent security ones that shouldn’t have been posted in that manner:
https://github.com/RsyncProject/rsync/issues/871
https://github.com/RsyncProject/rsync/issues/882
Seems like business as usual:
https://social.treehouse.systems/@thesamesam/116662824873341085
One thing I will say on rsync is that regressions aren’t new with it. It does something hard: it has to deal with symlinks. Releases have often had regressions for a long time, especially for security fixes, and it long predates LLMs, because symlinks are hard. Of course seeing a gazillion Claude commits still makes me uncomfortable, but it’s important to see what’s new as well.
In practice, they’re not very good because of broken FP16, broken kernels, high idle usage and a bunch of other things.
Same with the AMD MI50 and MI100. Looks great on paper, not practical IRL, unless you want to pay a whole team of software devs to fix them for you.
Better to just save up for a 2080 TI or 3090, sadly.
No.
Even the biggest open weights models are trained on pennies compared to OpenAI and Claude. They just don’t have the hardware to be so wasteful.
In fact, the Nvidia GPU ban was the best thing to ever happen to “small” AI devs. It made them thrifty.
MoEs can be very fast with hybrid inference. I run Xiaomi Mimo 2.5 (a 310B model, 116GB weights) on my single 3090 + 7800 CPU, and it outputs faster than I can read it.
It’s also easier to fit long context, if you need that.
It’s best to use the ik_llama.cpp fork for that, though. It gives a huge boost to hybrid MoE speeds.
Probably Qwen 35B then. ~9GB free VRAM + (let’s say) ~16GB of free CPU RAM is a good size for that, and squeezing bigger models in would be hard unless it’s a headless linux server.
Depends on how much CPU RAM you have, and how fast it is.
As others said, Qwen 35B at the very least. But you can get better models with more CPU RAM.
Yeah.
It’s not even about efficiency, really, but independence from corporations, privacy, and principle. Kind of like Lemmy.
This is a “feel guilty about missing recycling” kind of complaint.
Having a server run for an hour or two (?) a day is negligible. You use more energy running a fridge, or leaving a few lights on, or browsing Lemmy for a while. Or running a docker container for other services. You release more greenhouse gasses eating beef, or driving anywhere, or even opening your front door a few times, and individual industries are going to use vastly more electricity than a few self hosters ever would. If you own an EV, you’ve probably blown out your entire zip code of self hosters.
…But if it still bothers you, you can find an ewaste smartphone(s) and host on that. This is actually a very neat use case IMO.
However, if you get to the homelab scale of “an EPYC + 3090s running all the time” that electricity use does start to add up. But that’s quite a rare hobbyist tier, I’d say, and it really shouldnt be running 24/7.
As an observer in these comments, this is a great answer. Thanks for typing it out.
It does seem like some “cuts” could be ironed out reasonably quickly, like the file naming issue or UI lag.
I dunno. Psychologically, buyers don’t like an upfront tax. It’s just how our monkey brains are.
…How about an EV tax proportional to vehicle mass, or maybe battery mass/capacity? With a cutoff for tiny, cheap EVs. This would tax commercial vehicles and ridiculous luxury EVs like the Hummer EV, but incentivize small EVs or plug in hybrids. And it “makes sense” because heavy, long range vehicles wear roads more.
I assumed it would be wholesale prices. I guess cracking and refining oil yourself, or brewing biofuel, would be the analogue to backyard solar or whatever.
Part of the point is that “home gas” for local commutes and such is hilariously impractical, whereas that’s not the case for EVs.
How about an EV fast charger tax? Same as gas.
If you charge at home, it’s tax free. Just like a gas tank in your back yard would be, I assume.
I think Republicans would buy that too, as it’s still technically “EV negative” and not very visible to constituents (just like the gas tax).
That sounds better and worse. An attack could persist past one specific version without anyone noticing for a bit.
They’re specifically trying to emulate food people know (like burger patties). In that context, I think versioning makes sense: “this is our 1st attempt, our second is closer, our third is…”
And it’s something you buy if you specifically want emulation.
You aren’t wrong: direct, less processed diets are better. I’m just saying that this food is for a more narrow purpose.
Partially configured some parts via LLM but please don’t crucify me for that.
Slap in a spare GPU, and self-host one!
The 30B-class models are unbelievably good now, for being so small. They’re kinda where Claude was like a year ago, if not less. And (with the right backend) they aren’t expensive to host.
Briefly, Beyond Burger burger patties were cheaper in a (U.S.) Costco than the equivalent Morton’s patties.
It got me to try their v3 forumula… and its actually really good. You have to grill the snot out of it, but it’s good.
Very interesting read, thanks.
One common theme seems to be that “better is the enemy of good,” with (for example) pushback against mixing coal with waste for gasification, using cement plants or power plants for CO2, using biomass plant waste and such.
I see Bio/e-fuels as a “intermediate step” into the future, anyway, so I think trading a little sustainability for practicality is way better than nothing.