Never had the problems you mention till Delphi 7.
D4 had bugs in its COM code. In 2000 I worked at a Dotcom (homedirectory.com) and we used a D4 COM search component running in SQL Server 6.5. It used to crash when the CPU load went above 5%. I guessed it was D4, recompiled under D5 and that fixed it.
Microsoft as part of its settlement to Borland over the employee poaching (including AH) provided technical info that let D5 generate reliable COM code.
Delphi 1 was a mess. Delphi 2 was rock solid. Delphi 3 was a buggy mess. Delphi 4 kind of recovered things but was never as good as Delphi 2. By this point we'd figured out the pattern and skipped Delphi 5, if I recall correctly.
1 was 16 bits only. 2 turn to 32 bits AFAIK.