re pipelining: unless Go's network package does this magically under the hood, then sqlite-http doesn't take advantage of that! If you think that would be a nice feature, file an issue and I'll see what can be done
How complex is your text processing? If I'm working with a small dataset, I typically just save the "raw" responses into a table and create several intermediate tables that extracts the data I need. Something like:
create table raw_responses as http_get('https:/....');
create table intermediate1 as select xxxx(response_body) as message from raw_responses;
create table final as select message from intermediate;
(some CTEs[0] will be very useful here)The "lambda" pattern described about half-way through this post might be useful as well, if you need to do some complex text processing.