On Windows, once the file is open, it is that filename that is open; You can't rename or delete it; Therefore, if you want to replace a DLL (or any other file) that is in use, you have to kill any program that uses it before you can do that; And if it's a fundamental library everything uses (USER32.DLL COMCTL.DLL etc), the only effectively reliable way to do that is reboot.
On Unix, once you have a handle=descriptor of the file, the name is irrelevant; You can delete or rename the file, The descriptor still refers to the file that was opened. Thus, you can just replace any system file; Existing programs will keep using the old one until they close/restart, new programs will get the new file.
What this means is, that even though you don't NEED to restart anything for most upgrades in Unix/Linux, you're still running the old version until you restart the program that uses it. Most upgrade procedures will restart relevant daemons or user programs, or notify that you should, (e.g. Debian and Ubuntu do).
You always need a reboot to upgrade a kernel (kernel-splice and friends not withstanding), but otherwise it is enough in Unixes to restart the affected programs.
This is wrong... there's no clear-cut thing like the "file name" or "file stream" that you can specify as "in-use". It depends on the specifics of how the file is opened; often you can rename but not delete files that are open. Some (but AFAIK not all) in-use DLLs are like this. They can be renamed but not deleted. And then there's FILE_SHARE_DELETE which allows deletion, but then the handle starts returning errors when the file is deleted (as opposed to keeping the file "alive").
To make it even more confusing, you can pretty much always even create hardlinks to files that are "in use", but once you do that the new name cannot be deleted unless the old name can also be deleted (i.e. they follow the same rules). This should also make it clear that it's not the "name" that's in use, but the "actual file" (whatever that means... on NTFS I'd suppose it corresponds to the "file record" in the MFT).
The rule that Windows always abides by is that everything that takes up space on the disk must be reachable via some path. So you can't delete in-use files entirely because then they would be allocated disk space but unreachable via any path.
> What this means is, that even though you don't NEED to restart anything for most upgrades in Unix/Linux, you're still running the old version until you restart the program that uses it.
What I expect it also means is that you'll get inconsistencies when doing inter-process communication, since they'll be using different libraries with potential mismatches. Is this correct? Because it seems to me that the Windows method might be less flexible but is likely to be more stable, since there's a single coherent global view of the file system at any given time.
But keep in mind that if your system can't cope with this what you've done there is engineer in unreliability, you've made a system that's deliberately not very robust, unless it's very, very tightly integrated (e.g. two sub-routines inside the same running program) the cost savings had better be _enormous_ or what you're doing is just amplifying a problem and giving it to somebody else, like "solving" a city's waste problem by just dumping all the raw sewage into a neighbouring city's rivers.
Now, the "you can't delete things because then the disk space is unreachable" argument makes plenty of sense for, say, FAT, a filesystem from the 1980s.
But (present year argument) this is 2018. Everybody's main file systems are journalled. Sure enough, both systems _can_ write a record to the journal which will cause the blocks to be freed on replay and then remove that journal entry if the blocks actually get freed up before then. The difference is that Windows doesn't bother doing this.
> But keep in mind that if your system can't cope with this what you've done there is engineer in unreliability
It's weird that you're blaming my operating system's problems on me. "My system" is something a ton of other people wrote, and this is the case for pretty much every user of every OS. I'm not engineering anything into (or out of) my system so I don't get the "you've made a system that [basically, sucks]" comments.
> [other arguments]
I wasn't trying to go down this rabbit hole of Linux-bashing (I was just trying to present it as as objective of a flexibility-vs.-reliability trade-off as I could), but given the barrage of comments I've been receiving: I don't know about you, but it happens more often than I would like that I update Linux (Ubuntu) and, lo and behold, I can't really use any programs until I reboot. Sometimes the window rendering gets messed up, sometimes I get random error pop-ups, sometimes stuff just doesn't run. I don't get why it happens in every instance, and there might be lots of different reasons in different instances. IPC mismatch is my best guess for a significant fraction of the incidents. All I know is it happens and it's less stable than what you (or I) would hope or expect. Yet from everyone's comments here I'm guessing I must be the only one who encounters this. Sad for me, but I'm happy for you guys I guess.
Ubuntu developer here. This doesn't happen to me in practice. Most updates don't cause system instability. I rarely reboot.
Firefox is the most noticeable thing. After updating Firefox (usually it's a security update), Firefox often starts misbehaving until restarted. But I am very rarely forced to restart the login session or the entire system. I should, to get updates to actually take effect, but as a developer I'm usually aware of the specifics, so I can afford to be more selective than the average user.
Are you sure you aren't comparing apples to oranges here, and are actually complaining about the stability of updates while running the development release, which involves ABIs changing and so forth?
> You: All I can say is that, based on everything I know, that's not the current experience of the majority of users
>> Me: Yet from everyone's comments here I'm guessing I must be the only one who encounters this.
???
> You: It doesn't seem fair for you to generalize this to some architectural problem.
>> Me: I don't get why it happens in every instance, and there might be lots of different reasons in different instances. IPC mismatch is my best guess for a significant fraction of the incidents.
???
But we are the only 2 people in the world experiencing this, so never mind.
For systemd updates, I can just reload it. For the likes of core components like bash, and major DE updates, I can just lazily use loginctl to terminate all of my sessions, and start fresh.
I'm not sure why Firefox would be causing instability until you restart (reboot?), though.
Ideally they'd just prefork a template process for children and open fds for everything they need, that way such a detection wouldn't be necessary.
I get the impression that the UI loads files from disk dynamically, which start mismatching what was already loaded.
Lately, every time my co-worker has updated Ubuntu, it has broken his system. He's like my canary in the coalmine. I wait for his system to not fall over before I will update mine.
I don't know of a single Linux distro that works out of the box with my laptops. Maybe if I bought a $2,000 laptop that shipped with Linux it would work. It would still be a pain in the ass to update, though.
I kind of hate Linux as a desktop now. I've been using it as such for 14 years, and it's only gotten worse.
This is unfortunate. I've been using Linux and suffered, for the lack of a better word, with its warts since 2002.
There was a period between 2013-2016 where Linux was great as my main operating system. It was more stable than OS X and was much better for development.
Is hardware support your main issue with desktop Linux?
The software (especially Ubuntu's desktop) is lacking basic features from even 10 years ago. Maybe there's a way to get it to do what it used to do, but I can't figure it out, and I'm not going to research for two days to figure it out. I just live with a lack of functionality until I can replace this thing.
Not only that, but things are more complicated, with more subsystems applying more constraints (in the name of compatibility, or security, or whatever) that I never asked for and that constantly gets in my way. Just trying to control sound output and volume gives me headaches. Trying to get a new piece of software to work requires working out why some shitty subsystem is not letting the software work, even though it is installed correctly. Or whining about security problems. You installed the software, Ubuntu, don't fucking whine to me that there's a SELinux violation when I open my browser!
Hardware is a big problem because modern software requires more and more memory and compute cycles. All of my old, stable laptops can no longer perform the web browsing workloads they used to. Browsers just crash from lack of memory, or churn from too much processing. If you don't use modern browsers, pages just won't load.
Aside from the computing power issue, drivers are garbage. Ignoring the fact that some installers simply don't support the most modern hard disks, and UEFI stupidity, I can't get video to work half the time. When I can, there are artifacts everywhere, and I have to research for three days straight to decipher what mystical combination of graphics driver and firmware and display server and display configuration will give me working graphics. Virtually every new laptop for several years uses hybrid graphics, and you can't opt-out or you get artifacts or crashing. Even my wifi card causes corruption and system crashing, which I can barely control if I turn off all the features of the driver and set it to the lowest speed! Wifi!!! How do you screw that up, seriously?
Modern Linux is just a pain in the ass and I'm way too old to spend my life trying to make it work.
Hardware support wise, newer kernels generally come to Debian sooner too, as the latest stable kernel generally gets into Sid a week or so after release, then added to backports for Debian Stable after a few weeks. Currently you can nab 4.14 from backports on Debian Stable, and 4.15 should be coming down the pike shortly (seeing as its just a few days old).
http://neugierig.org/software/chromium/notes/2011/08/zygote....
Regarding the rest, if your code has incompatible API breaks between two patch or minor version changes, you’ll need to rethink your development model.
Which is almost true. In fact, you were unable to use programs that changed runtime dependencies or conflicted with current user sessions, init processes or kernel modules. You can often use other programs, but not ones that in any way touched the ones you upgraded, for one reason or another.
If you have to upgrade, say, a command line utility, that almost always doesn't require rebooting. If you have to upgrade a GUI app, or a tool that depends on some bastardized unholy subsystem designed to "secure desktop sessions", that may very well require relinquishing the session and restarting it. If you have to upgrade a tool used by your desktop (and if you have a complex desktop, that is literally thousands of programs), it's the same story, though you may even need to restart your desktop session manager or even your display server.
Then there's system init processes, kernel modules, firmware, system daemons and the like. You can reload those without rebooting, but it's certainly not easy - you will probably have to change to runlevel 1, which kills almost everything running. You can reload the kernel without rebooting, too - very handy for live patching - but really, why the hell would anyone want to do this unless they were afraid to power off their system?
So, technically, rebooting is not required to update in many cases in Linux, just like in Windows. But it is definitely the simplest and most reliable way.
Thanks, I'm glad at least one person agrees I'm not hallucinating. The vast majority of people here are telling me I'm basically the only one this happens to.
Monday's I merge last week's snapshot, take a new one, and run all my updates. Then I do my dev work in my VM. Before I head out on trips, I just ship the entire machine over the network to my MacBook Pro.
This is mostly because have you literally ever tried to install any Linux on laptops? It's always a Russian roulette with those $+#&ing Broadcom wireless chipsets. >.<
So you're not hallucinating. Linux as a desktop/laptop had a sweet spot from like...2012-ish till 2016. Then 802.11ac went mainstream so Broadcom released new chipsets and graphics cards had a whole thing with new drivers and Ubuntu's packagers (the people) lost their mind or something.
Nothing feels right, at least in Ubuntu/Arch land right now.
This is less of an issue with Linux, per se, and more to do with proprietary video drivers.
I have multiple systems in my home with various GPUs. The systems running Intel and AMD GPUs with open source drivers don't have this problem. The two desktops with Nvidia GPUs have this problem whenever the Nvidia driver is updated.
I also had the same exact problem with my AMD system back when it was running the proprietary fglrx driver.
Actually, it IS a problem with Linux. I don't get this behavior on my Windows or OSX machines where NVIDIA has been reliably (modulo obvious caveats) shipping "evil proprietary" drivers for a decade.
Linux is great, but it doesn't need to be coddled.
No one is blaming you specifically, it is a common way of saying, “if you write operating systems, and you do $THING, you will get $RESULT.” Common, but wrong, which is why your high school English teacher will ding you for phrasing something that way.
I just don't update nvidia or my kernel automatically and magically I only have to reboot less than once a month and always on my schedule.
Windows patches are a much bigger pain in the ass to deal with on a month-to-month basis, but Linux patches can really bite you.
Example 1:
Say I have an application 1 that uses shared library X, and a application 2 that spawns an external process every 5 minutes that uses library X and communicates in some way with application 1. Now let's say that library X v2.0 and v2.1 are incompatible, and I need to apply an update.
On Windows, if I update this program, it will keep running until the system is rebooted. Updates, although they take significant time due to restarts, are essentially atomic. The update either applies to the entire system or none of the system. The system will continue to function in the unpatched state until after it reboots.
On Linux, it's possible for application 1 to continue to run with v2.0 of the shared library, while application 2 will load v2.1, and suddenly your applications stop working. You have to know that your security update is going to cause this breaking change and you need to deal with it immediately after applying the update.
Example 2:
A patch is released which, unbeknownst to you, causes your system to be configured in a non-bootable state.
On Windows, you'll find out immediately that your patch broke the system. It's likely (but not certain) to reboot again, roll back the patch, and return to the pre-patched state. In any event, you will know that the breaking patch is one that was in the most recently applied batch.
On Linux, you may not reboot for months. There may be dozens or hundreds of updates applied before you reboot your system and find that it's not in a bootable state, and you'll have no idea which patch has caused your issue. If you want your system in a known-working state, you'll have to restore it prior to the last system reboot. And God help you if you made any configuration changes or updates to applications that are not in your distro's repository.
Unix semantics were IIRC in place as far back as v7 (1979), possibly earlier - granted, a PDP disk from that time was bigger (~10-100MB) than the corresponding PC disk from a few years later (~1-10MB), but an appeal to technological progress in this particular example case is a moot point.
Only if the libraries that use IPC have changed their wire format between versions, which would be a pretty bad practice, so I wouldn't expect that to happen often (if ever).
If something that's already running has its data files moved around or changed sufficiently, and it later tries to open (that is, the app was running but the data file wasn't open when the upgrade happened) what it thinks is an old data file, but is either new and different or just missing, that could cause problems.
> Because it seems to me that the Windows method might be less flexible but is likely to be more stable, since there's a single coherent global view of the file system at any given time.
In practice I've never had an issue with this (nearly 20 years using various Linux desktop and server distros). Upgrade-in-place is generally the norm, and most people will only reboot if there's a kernel update or an update to the init system.
In theory, but Linux systems tend to do very little IPC other than X11, pipelines, and IP-based communication, where the protocols tend to support running with different versions.
In practice you can achieve multi-year uptimes with systems until you get a mandatory kernel security update.
While I find the situation different with systemd maintainers, whose communication style used to be questionable too often, and their software had some nasty bugs, I must admit that despite those problems they've also built software, which is nevertheless reliable, even though it took them a lot of time to come there.
This, honestly, is the most disappointing statement I read from you in the recent couple of years. And I'm saying this as a person who used to respect you a lot. I find your lack of respect together with the willingness to spread lies like this quite appalling.
I've primarily been using AwesomeWM for the last few years and occasionally XFCE (both on ArchLinux) and I cant recall ever experiencing what you describe.
If my system is closed to the public world, has a tiny amount of external services, and I am aware of the specific bug delta since system release and what mitigations may or may not be required, I can leave it running as long as I choose to accept the risk. Cute phrases like 'pattern' and 'anti-pattern' are rules of thumb, not absolute truths.
It's a really common engineering task to do this and I'm not at all surprised that someone trying to maintain uptime would do so. Honestly it's more mature than updating every time because each change also introduces more potential for regression. If your goal is to run a stable system you want to avoid this unless the risk is outweighed.
kgraft for example swaps off each syscall for a process while it is not being used. This lets the OS slowly transfer to the new kernel as it is running.
kpatch does it all in one go but locks up the system for a few milliseconds.
The version that is currently merged into 4.0+ kernels is a hybrid of the two developed by the authors of both systems.
While this is true, I've never seen this to be a problem. If two programs use IPC, they usually use either stable, or compatible protocol.
To make things even more complicated, you can have two programs, either each in it's own container, or statically linked, or with their private bundles of libraries, doing IPC and then they are free to have different versions of the underlying libraries, while the users still expect them to work fine.
At the first glance this is true, but you can guard against this in several ways. If your process only forks children then it already inherits the loaded libraries from the parent as part of the forked address space. Alternatively you can pass open file descriptors between processes. Another option is to use file-system snapshots, at least if the filesystem supports them.
Yet another option is to not replace individual files but complete directories and swap them out via RENAME_EXCHANGE (an atomic swap, available since kernel 3.15). As long as the process keeps a handle on its original working directory it can keep working with the old version even if it has been replaced with a new one.
Some of those approaches are tricky, but if you want to guard against such inconsistencies at least it is possible. And if your IPC interfaces provide a stable API it shouldn't be necessary.
> And then there's FILE_SHARE_DELETE which allows deletion
That has some issues when the file is mmaped. If I recall correctly you can't replace it as long as a mapping is open.
Ah. This must be why I can't permanently delete files in use by a program but can sometimes "delete" them and send them to the recycle bin.
That's why you should restart those programs that were using the library. You can find this out via `lsof`.
1. Do the upgrade, this changes the files.
2. You have a log of what packages got upgraded.
3. Run the package system's introspection commands to see what files belonged to that package before & after
4. Run the package system's introspection commands to see what reverse depends on those packages, or use the lsof trick mentioned upthread.
5. For each of those things restart any running instance of the application / daemon.
Even if something didn't implement this already (most mature systems to) this is at most a rather trivial 30 line shellscript.
Some versions are known to be incompatible and most Linux distributions do a very good job of recommending and doing a restart of affected services in a way transparent to users. I have been running Linux and home and at work for years, almost never restart those workstations and, as far as I can tell, never had problems from piecemeal upgrades. My 2c.
Isn't there a $MFT\\{file entry number} virtual directory that gives an alternate access path to each file? Wouldn't that qualify as "a way to access the file?"
Also, you might say that in practice Linux abides by the same rule - the old file can be referenced through some /proc/$pid/fd/$fd entry.
I’m not sure what scenario you are envisioning. Usually upgrades are handled via the distribution and its package manager, and the maintainers take care of library issues. It’s not like windows where you go all over downloading packages from websites and installing them over each other.
>In a case where one could get some sort of inconsistency because of different library versions, you restart the applications.
I am envisioning the same scenario you replied to!
In any case, my assumption here is we're trying to help the user and give them an easy way to know what to do, instead of leaving their software in an undefined state.
[1] Note that is not the same as your "must be reachable via some path". It is literally inaccessible by name after delete. Try to access by name and you get STATUS_DELETE_PENDING. This is unrelated to the other misfeature of being able to block deletes by not including FILE_SHARE_DELETE.
There is also an API that retrieves a filename from a handle. I don't think it guarantees the name be usable though.
It's easy to imagine a system that works the way I would have it, because it exists: Unix. You can unlink and keep descriptors open. NT is very close to being there too, except for these goofy quirks which are kind of artificial.
You can rename a file when it's open. And once it's renamed, you can delete it.
Of course there is a theoretical possibility that this will happen; however, in practice, updates (especially security updates) on Linux happen with ABI compatible libraries. E.g. on debian/ubuntu
apt-get update && apt-get upgrade
Will generally only do ABI compatible updates, without installing additional packages (you need 'dist-upgrade' or 'full-upgrade' for that).
Some updates will go as far as to prevent a program restart while updating (by temporarily making the executable unavailable).
Firefox on Ubuntu is an outlier - an update will replace it with one that isn't ABI compatible. It detects this and encourages you to restart it.
All in all, it's not that a reboot is never required for linux theoretically - it is that practically, you MUST reboot only for a kernel update, and may occasionally need to restart other programs that have been updated (but are rarely forced too).
It's small details like these that make so much difference in daily convenience.
Renaming/Deleting files in use is one of those things that us nerds like to complain about, but it makes sense when you think of an accountant that has an open spreadsheet and accidentally deletes a folder with that file. For average non-technical people (on any OS) I would say it makes sense to block that file from being deleted.
I've had this happen in bash also, where I modify some script in an external editor then try to run it, only to realize that I'm running from the trash, even though the bash prompt naively tells me I'm in ~/SomethingNotTrashFolder.
Intuitive would be "Hey, this file you're working on was just moved to the trash. There's a 99.9999% chance you don't want to do this." rather than hoping the filepath is visible, and noticed by dumb chance, in the title bar, since not many people periodically glance at the file they have open to verify it's still the file they want open.
I am not sure about deletion, but one of my programs has been using the ability to rename itself to do updates for the last 15 years.
It's simple for any application to open a file in Windows such that it will allow a rename or delete while open - set the FILE_SHARE_DELETE bit on the dwShareMode arg of the win32 CreateFile() function. In .NET, the same behaviour is exposed by File.Open / FileShare.Delete.
Can't speak for everyone else, but Windows fully supports shared file-access which prevents the kind of file-locks which causes reboot requirements.
The problem is that the default file-share permissions in the common Windows APIs (unless you want to get verbose in your code) is that the opening process demands exclusive access and locking to the underlying file for the lifetime of that file-handle.
So unless the programmer takes the time to research that 1. these file-share permissions exists, 2. which permissions are appropriate for the use-cases they have in their code, and 3. how to apply these more lenient permissions in their code...
Unless all that, you get Windows programs which creates exclusive file-locks, which again causes reboot-requirements upon upgrades. Not surprising really.
In Linux/UNIX, the default seems to be the other way around: Full-sharing, unless locked down, and people seem prepared to write defensive code to lock-down only upon need, or have code prepare for worst-case scenarios.
There is a registry key which allows an update to schedule a set of rename operations on boot to drop in replacement file(s). https://blogs.technet.microsoft.com/brad_rutkowski/2007/06/2...
This is not always correct; see: https://news.ycombinator.com/item?id=16256483
We are getting there on Linux too - with atomic or image based updates of the underlying system. On servers you will (or already) have A/B partitions (or ostrees), on mobiles and IoT too, some desktops (looking at Fedora) also prefer reboot-update-reboot cycle, to prevent things like killing X while doing your update and leaving your machine in inconsistent state.
macOS also does system updates with reboot, for the same reasons.
https://blogs.gnome.org/hughsie/2012/06/04/offline-os-update...
https://www.freedesktop.org/wiki/Software/systemd/SystemUpda...
https://fedoraproject.org/wiki/Features/OfflineSystemUpdates
It's still much faster than a Windows update (at least on my SSD system).
And speaking from experience, the recent macOS way of applying an update is absolutely insane - on my Macbook Pro (with the stock SSD with 3GB/sec read and 2GB/sec of write) a small system update can take 10+ minutes to install
At least on *nix, even when you need a reboot, once is enough.
Historically (Windows 95 and earlier) reboots were required to reload DLLs and so on but that’s not really true anymore. Still a lot of installers and docs say to reboot when it’s not really necessary as a holdover from then
No it is exactly the same principle: something has changed therefore invalidate all existing contexts. Far less error prone than trying to recompute them, what happens e.g. if a resource has already been accessed in a context that is now denied? Security 101.
As for logging other places out, that's a design choice. People change password either because they routinely change theirs (they either need to or choose to), or because of a (suspected) compromise. In the latter case you'll probably want to log everyone else out (though, who says you're logging out the attacker and not the legitimate user?) and in the former case you shouldn't (otherwise changing your password becomes annoying and avoided). The interface for changing the password could have a "log out all sessions" checkbox or it could just be a feature separate from changing your password.
No, it's not as simple as you put it. No need to condescendingly pass it off as "security 101".