goes looking for the issue

PostgresSQL has a limit of 65,535 parameters, so bulk inserts can fail with large datasets.

Hmm. I would believe that there are efficiency gains from doing one large insert rather than many small — like, there are probably optimizations one can take advantage of in rebuilding indexes — and it’d be nice for database users to have a way to leverage that.

On the other hand, I can also believe that DBMSes might hold locks while running a query, and permitting unbounded (or very large) size and complexity queries might create problems for concurrent users, as a lock might be held for a long time.

EDIT: Hmm. Lock granularity probably isn’t the issue:

https://stackoverflow.com/questions/758945/whats-the-fastest-way-to-do-a-bulk-insert-into-postgres

One way to speed things up is to explicitly perform multiple inserts or copy’s within a transaction (say 1000). Postgres’s default behavior is to commit after each statement, so by batching the commits, you can avoid some overhead. As the guide in Daniel’s answer says, you may have to disable autocommit for this to work. Also note the comment at the bottom that suggests increasing the size of the wal_buffers to 16 MB may also help.

is worth mentioning that the limit for how many inserts/copies you can add to the same transaction is likely much higher than anything you’ll attempt. You could add millions and millions of rows within the same transaction and not run into problems.

Any lock granularity issues would also apply to transactions.

Might be concerns about how the query-processing code scales.

Do you want the router to also be 10" rack-mountable? That seems like it’d be a big input into the hardware you get.

Are you looking for specific values in some field in this table, or substrings in that field?

If specific values, I’d probably import the CSV file into a database with an column indexed on the value you care about.

I’m not familiar enough with Cloudflare’s error messages — or deployment with Cloudflare — to know what exact behavior that corresponds to, but I’d guess that most likely it can open a TCP connection to port 443 on what it thinks is your server, but it’s not getting HTTPS on that port or your server isn’t configured to serve up the right certificate for that hostname or the web server software running on it is otherwise broken. Might be some sort of intervening firewall.

I don’t know where your actual server is, may not even be accessible to me. But if you have a Linux machine that can talk to it directly – including, perhaps, the server itself – you should be able to see what certificate it’s handing back via:

$ openssl s_client -showcerts -servername akaris.space IP-address-of-actual-server:443

That’ll try to establish a TLS connection, will send the specified server name so that if you’re using vhosting on the server, it knows which site to return, and then will tell you what certificate the web server used. Would probably be my first diagnostic step if I thought that there was a problem with the TLS handshake on a machine I was running.

That might provide enough information to you to let you resolve the issue yourself.

Beyond that, trying to provide much more information probably isn’t possible without more information about how your server is set up and what actually is working. You can censor IP addresses if you want to keep that private.

kagis

https://en.wikipedia.org/wiki/Mr._Men

Mr. Men is a British series of children’s books and media franchise written and illustrated by English author Roger Hargreaves which began publication in August 1971.[1] From 1981, an accompanying series of Little Miss books by the same author (but with female characters) was published.

The timeframe is plausible; they were out then.

I have been known to use XMODEM over screen over telnet for file transfer.

THE FUTURE IS TERMINAL!

The past and present are also terminal.

I’m sorry, you are correct. The syntax and interface mirrors docker, and one can run ollama in Docker, so I’d thought that it was a thin wrapper around Docker, but I just went to check, and you are right — it’s not running in Docker by default. Sorry, folks! Guess now I’ve got one more thing to look into getting inside a container myself.

While I don’t think that llama.cpp is specifically a special risk, I think that running generative AI software in a container is probably a good idea. It’s a rapidly-moving field with a lot of people contributing a lot of code that very quickly gets run on a lot of systems by a lot of people. There’s been malware that’s shown up in extensions for (for example) ComfyUI. And the software really doesn’t need to poke around at outside data.

Also, because the software has to touch the GPU, it needs a certain amount of outside access. Containerizing that takes some extra effort.

https://old.reddit.com/r/comfyui/comments/1hjnf8s/psa_please_secure_your_comfyui_instance/

ComfyUI users has been hit time and time again with malware from custom nodes or their dependencies. If you’re just using the vanilla nodes, or nodes you’ve personally developed yourself or vet yourself every update, then you’re fine. But you’re probably using custom nodes. They’re the great thing about ComfyUI, but also its great security weakness.

Half a year ago the LLMVISION node was found to contain an info stealer. Just this month the ultralytics library, used in custom nodes like the Impact nodes, was compromised, and a cryptominer was shipped to thousands of users.

Granted, the developers have been doing their best to try to help all involved by spreading awareness of the malware and by setting up an automated scanner to inform users if they’ve been affected, but what’s better than knowing how to get rid of the malware is not getting the malware at all. ’

Why Containerization is a solution

So what can you do to secure ComfyUI, which has a main selling point of being able to use nodes with arbitrary code in them? I propose a band-aid solution that, I think, isn’t horribly difficult to implement that significantly reduces your attack surface for malicious nodes or their dependencies: containerization.

Ollama means sticking llama.cpp in a Docker container, and that is, I think, a positive thing.

If there were a close analog to ollama, like some software package that could take a given LLM model and run in podman or Docker or something, I think that that’d be great. But I think that putting the software in a container is probably a good move relative to running it uncontainerized.

PNG has terrible compression

It’s fine if you’re using it for what it’s intended for, which is images with flat color or an ordered dither.

It’s not great for compressing photographs, but then, that wasn’t what it was aimed at.

Similarly, JPEG isn’t great at storing flat-color lossless images, which is PNG’s forte.

Different tools for different jobs.

At least at one point, GIF89a (animated GIF) support was universal among browsers, whereas animated PNG support was patchy. Could have changed.

I’ve also seen “GIF” files served up online that are actually, internally, animated PNG files, so some may actually be animated PNGs. No idea why people do that.

On the “better compression” front, I’d also add that I doubt that either PNG or WebP represent the pinnacle of image compression. IIRC from some years back, the best known general-purpose lossless compressors are neural-net based, and not fast.

kagis

https://fahaihi.github.io/NNLCB/

These guys apparently ran a number of tests. They had a neural-net-based compressor named “NNCP” get their best compression ratio, beating out the also-neural-net-based PAC, which was the compressor I think I recall.

The compression time for either was far longer than for traditional non-neural-net compressors like LZMA, with NNCP taking about 12 times as long as PAC and PAC taking about 127 times as long as LZMA.

What’s next?

I know you all immediately wondered, better compression?. We’re already working on that. And parallel encoding/decoding, too! Just like this update, we want to make sure we do it right.

We expect the next PNG update (Fourth Edition) to be short. It will improve HDR & Standard Dynamic Range (SDR) interoperability. While we work on that, we’ll be researching compression updates for PNG Fifth Edition.

One thing I’d like to see from image formats and libraries is better support for very high resolution images. Like, images where you’re zooming into and out of a very large, high-resolution image and probably only looking at a small part of the image at any given point.

I was playing around with some high resolution images a bit back, and I was quite surprised to find how poor the situation is. Try viewing a very high resolution PNG in your favorite image-viewing program, and it’ll probably choke.

• At least on Linux, it looks like the standard native image viewers don’t do a great job here, and as best I can tell, the norm is to use web-based viewers. These deal with poor image format support support for high resolutions by generating versions of the image at multiple pre-scaled levels and then slicing the image into tiles, saving each tile as a separate image, so that a web browser just pulls down a handful of appropriate tiles from a web server. Viewers and library APIs need to be able to work with the image without having to decode the whole image.

  gliv used to do very smooth GPU-accelerated panning and zooming — I’d like to be able to do the same for very high-resolution images, decoding and loading visible data into video memory as required.

• The only image format I could find that seemed to do reasonably well was pyramidal TIFF.

I would guess that better parallel encoding and decoding support is likely associated with solving this, since limiting the portion of the image that one needs to decode is probably necessary both for parallel decoding and for efficient high-resolution processing.

WebP had been kind of moving in on its turf, based on what I’ve been seeing websites using.

I assume that this is an attempt to look for brands that run on the small or large side and compensate for it by comparing how customers buy that size relative to the sizes they buy with other brands.

They can probably recommend your personal size based on other purchases that you’ve made.

There are ISO country codes.

https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes

Cat people. Filthy cat people, pushing the cat agenda.

The last time I used a commercial VPS, I’m pretty sure it used VNC to provide console access.

The VNC software I linked to above appears to support TLS. If TLS isn’t sufficient transport security, then most Internet-using software is going to be in trouble.

I’m not sure what you mean by subjective.

I haven’t looked at the VNC protocol for a while, but I don’t think that it imposes any terrible inefficiencies. A couple of decades back, I needed to implement something quick-and-dirty similar to VNC, and went with rendering window contents and handling dragging of windows locally, which I don’t believe that VNC can do (or didn’t then) but IIRC VNC has a tile cache, which, if intelligently used, should avoid most traffic. Dunno if it can deal well with efficiently rendering visual effects.

VNC is dead.

How so?

There are a number software packages in Debian that implement VNC. To grab one random example, the last commit to their git repo was last month.

If OP is specifically wanting commercial support, they do mention two companies that provide services.

https://www.freerdp.com/support

FreeRDP itself is a community driven open source project. However there are some people and companies that do offer all sorts of commercial support around FreeRDP:

• David Fort - Hardening Consulting - Contact
• Thincast Technologies GmbH - Contact

FreeRDP doesn’t seem to be soliciting donations, though.