xk3

You are a kind human being. Your passion is overwhelming and I wish you great success. Time to abolish the tyranny of paperwork.

https://unli.xyz/city/honeymoon/

The passport compatibility information is automatically refreshed on a daily basis.

There are some UI quirks which take a minute or two to figure out and I'm not going to change it. The search box is somewhat unconventional because making a list of tens of thousands of cities is not performant or easily navigable so you have to search and then filter the results.

I wrote a tool that might help with this if you don't want to download the files immediately but it definitely isn't required if you just download everything and use yt-dlp's download-archive.

    pip install xklb
    lb tubeadd podcasts.db 'http://www.hellointernet.fm/podcast?format=rss'
    lb listen podcast.db  # it will keep track of what you played and prioritize playing things that haven't been played before

    lb tubeupdate podcasts.db

    lb listen podcast.db -u time_modified

    lb dl podcast.db --audio  # with the above example it saves to the folder "Generic" in the current working directory since it uses the yt-dlp generic extractor

> I've been reluctant to put this app in front of others even though it is fully functional

I have a few projects like that. I've built something that works for me but I don't really want to bare the maintenance costs so I don't tell anyone about it

You should definitely consider adding save to / load from localStorage, IndexedDB, or URL.

Nice thing about URL is you can send pages to other people like this:

https://unli.xyz/tabsender/?links=%22open%20a%20bank%20accou...

Or use LZString https://github.com/pieroxy/lz-string like this:

https://unli.xyz/html/edit.html?t=%22DwBwfALghgTgXlAdsA9OYAz...

An accidental page refresh shouldn't eliminate all your TODOs :/

- EPSG:3857 Web Mercator

- EPSG:6933 https://nsidc.org/data/user-resources/help-center/guide-ease...

- UTM

You could just support Web Mercator, if normal or swapped are not valid coordinates, assume Web Mercator and check if they are valid coordinates. If they are swapped it will still be in the general area even if the projection is not exactly matching

I agree it is pretty niche. It might be nice to support non-WGS84 coordinates

https://tildes.net/~comp/19wc/what_programming_technical_pro...

small world

  Location: Illinois, USA
  Remote: Yes
  Willing to relocate: No
  Technologies: Python, PostgreSQL, PostGIS, GDAL, OGR, BigQuery, QGIS
  Résumé/CV: unli.xyz/resume.pdf
  Email: chapmanjacobd <at> gmail

Cluster-sort will bubble up unique groups first. It is sorted such that KMeans groups with fewer entries will appear first and within each KMeans group the paths / lines will be sorted alphabetically. (you can use the `--groups` flag to take a peak at the groupings)

I maintain a few line delimited files of musicians to look up, movie recommendations from different country subreddits, as well as browser tabs that I failed to read during the week. I have a simple cronjob script to automatically open some of these tabs at various points throughout the day.

Cluster-sort is really good at sorting these types of files because even if the band name is in the middle of a sentence it will be grouped with similar ones (If the lines of text were exact duplicates I would just use `sort --unique` to dedupe them). Having the lines grouped together prevents me from googling the same things every month--but maybe the downside of this approach is no spaced repetition (SRS) and less of a chance to "acquire taste".

    progress -wc ffmpeg

[0] https://github.com/Textualize/textual/discussions/228

[1] https://github.com/chapmanjacobd/library

ThreadPoolExecutor if IO-bound

ProcessPoolExecutor if CPU-bound

for example

    from concurrent.futures import ThreadPoolExecutor

    with ThreadPoolExecutor(max_workers=4) as e:
        e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
        e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
    print("everything done!")

Even if you only look at summary statistics for number of files or number of lines changed it makes it really obvious when some pragram is acting up or doing something weird. And there have already been a handful of times where, because reverting a change was only a `git restore x` away, I have saved several minutes of time. If you have snapshots that's good but being able to quickly use them is better than just knowing they exist.

    cat .xbindkeysrc
    "xdotool click 1"
      Alt_R
    "xdotool click 3"
      Menu

    xset -r 108
    xset -r 135
    pkill xbindkeys && xbindkeys

ProcessPoolExecutor if CPU-bound

for example

    with ThreadPoolExecutor(max_workers=4) as e:
        e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
        e.submit(shutil.copy, 'src2.txt', 'dest2.txt')

you might be interested in converting the pushshift data to parquet. Using octosql I'm able to query the submissions data (from the begining of reddit to Sept 2022) in about 10 min

https://github.com/chapmanjacobd/reddit_mining#how-was-this-...

Although if you're sending the data to postgres or BigQuery you can probably get better query performance via indexes or parallelism.

    "xdotool mousedown 1"
      b:49+1
    "xdotool mouseup 1"
      b:49+Release+1

You feel that the consumption is shallow yet you get value from it. Your work is tedious. If your desire is to do some work-like activities after work but you feel that you don't have the mental energy, you should try some activities like physical exercise, naps, or cold showers.

https://www.betterup.com/blog/mental-fatigue

If you're just feeling guilty pleasure from the Internet variety of "watching TV" I'd say get over it. You deserve it even if you've only been productive one minute today but your job is stressful.

If you want to do a different activity but you can't stop your behavior then you are addicted. In this case you should try to change your environment so that it becomes easier to do the thing you want to do and much harder to do the activity that you want to do less. If this doesn't work you might need professional assistance but that's not something to be ashamed of

My passion project is building CLI tools for managing files and curating media. I've been at it about 2 years now.

Most recently I wrote a subcommand for balancing files between disparate disks. Useful for mergerfs filesystems

Do you mean partially intact? I did not count that data. Ah I think I understand what you're saying. 78% of files are fully readable (by number of files) but most of those are small files and those are stored within btrfs metadata ("inlined" extents)

I computed 26% using the quantity of data (ie. sum() of file size) which I feel is a more accurate representation of what is readable. Sure, btrfs `single` mode will have many partially readable files--and if that is an ideal failure state then I would recommend it.

I tried the same experiment with raid0 btrfs config and only the inlined extents were fully readable--less than 1MB recovered from 80GB of data.

> you would expect to lose about 25% of files, while the remaining 75% would be intact

That's what I expected from btrfs for that last 9 months and people online were saying that `single` mode is the same data guarantees as `raid0` mode--which is kind of true but also it is kind of not as we can see. It's true but not likely that in a highly fragmented filesystem the spread of data in `single` could be a similar shape to `raid0` and in that case you could only easily recover the same amount of data (almost none).

What happens in practice is that btrfs will allocate 1gb blocks one drive at a time but, in a multi-disk setup, it writes file extents to multiple disks at a time. So at the file level there are no guarantees about one file being on one disk. This is why I was only able to read 20~30% of data rather than the 75% you and I both naively expected from btrfs single mode. It's important to note that this 20~30% is not guaranteed--it depends how file extents are saved across multiple disks and that is probabilistic not deterministic.

> the remaining 75% would be intact (especially if the files are small enough)

If all the files are inlined extents (default limit is 2048 bytes per file), and you were using raid1c4 metadata profile, then theoretically you could have 100% intact even after losing 3 of 4 disks (regardless of what the data profile is set to since that would not be used to save the file data)--but you would be using 80GB of allocated as "metadata" space in btrfs. (I have not tested that scenario but I think it is likely to be true). So all of the file redundancy is provided by the raid1c3 metadata configuration which I used for <2kb files in my test but the larger files like the max() 2GB one were recovered due to the chance that the file extents were only saved on one or more of the other three disks.

> Although I can see that on your latter experiment, you've defragged the btrfs filesystem beforehand

Yes, I think btrfs defrag does not do much different from when it writes the files initially, but it is still a useful utility in situations where files were overwritten many times. As I understand it there are many reasons that btrfs will decide to write a file to multiple extents and there seems to be no option to have it write one file to one disk as much as possible

> all of your files could be allocated on the first disk only

maybe a good example would be if I had filled up a disk then added a new one. btrfs really tries to allocate data fairly but adding a new disk is a situation where it would definitely be skewed toward one disk. I was actually thinking of recreating my filesystem and just copying over data one disk at a time so that the file extents would be written more consolidated to each disk--but still there would be no mechanism to prevent cross-disk extent writing...

> But on the other hand, if you'd lose one of the other disks, you might not lose any data whatsoever

yep

> even if free space was not fragmented

That would be ideal but I think it is pretty unlikely in practice unless the 1gb blocks which btrfs allocates per disk are used immediately and no files are appended to or changed then there is lots of free space within each 1gb block for btrfs to find

> Unfortunately, ... I can't give you more insight than this

Your comments were helpful and interesting. Hopefully I could share some of my findings as well. I still like btrfs but it certainly acts like a mad chef who is trying to boil 6,827 pots of water to cook spaghetti in this situation.

multi-disk and `single` profile is a bit weird. I'm planning on switching my array to individual btrfs `single` profile disks with `dup` metadata. I will also try MergerFS to group them into one disk but if MergerFS feels sketchy I'll just interface directly with many disks and balance files between them manually

Yes, this is what I'm starting to discover also. Because single mode is the default I assumed for the last 9 months that it would provide similar guarantees as MergerFS. But that was my incorrect assumption. As a result of this experiment I'm investigating MergerFS and I will likely switch from multi-disk btrfs to an array of single-disk btrfs under MergerFS.

edit: actually I'm not sure I would convert my array to raid0 because the speeds I'm getting are okay. 20% of data is better than nothing. I did the same with raid0 data profile and only inlined extent files were readable