Yes, extensions will eventually eat up a few gigabytes, but the is so small that you can keep the installer on a Google Drive link, a Dropbox folder, or even an old floppy disk (if you chain 70 of them together).
The CachedExtensionVSIXs folder. VS Code keeps downloaded extensions here. You can safely delete old versions if you need space. How to check your actual VS Code size Windows (PowerShell): visual studio code download size
How Big is VS Code? A Deep Dive into the Download & Install Size Yes, extensions will eventually eat up a few