An OpenVZ Experiment - How many containers?


I was wondering just how many OpenVZ containers I could create on a beefy machine and how many processes the Linux kernel would be happy running so I decided to do an experiment.

I have two OpenVZ hosts... one is the primary and the other is a backup machine. Both of them are HP Proliant DL380 Gen5 machines with dual, quad-core Xeon processors, 32GB of RAM, 32GB of swap, and a 600GB /vz partition. I decided to use the backup OpenVZ machine for the experiment.

I thought I'd start with a very high target of 1,000 containers and see how far I got... so I used vzsplit to generate a sample config (vzsplit -n 1000 -f split1000). Then I wrote a script to create 1,000 containers, one at a time. The script uses the centos-5-i386-default OS Template (124MB .tar.gz file), assigns an IP address and DNS, then starts up the container. Between each container creation it sleeps for 60 seconds... to give the container processes a little time to start.

Here is what one iteration of the script looked like:

vzctl create 1001 --ipadd --hostname container1001.localdomain \
--config split1000 --ostemplate centos-5-i386-default ;
vzctl set 1001 --nameserver --save ;
vzctl start 1001 ;
sleep 60 ;

The load on the machine usually stayed at 3 or lower during the entire creation process... which took a while given the amount of time to extract the OS Template, the minute pause, and the large number of containers I was creating.


I was able to create and start 638 containers. It was able to create more containers than that but at container 639 it couldn't start up any more. Here is the error message I got:

Dec 21 12:51:57 openvz2 kernel: Can't switch to fairsched node 1639
Dec 21 12:51:57 openvz2 kernel: CT: 1639: stopped
Dec 21 12:51:57 openvz2 kernel: CT: 1639: failed to start with err=-12

Update: - 9DEC09 - One of the OpenVZ/Parallels engineers provided a work around to this problem but I didn't get around to trying it out until a year later to discover that it worked. See, "Experiment 1 year later" and "Fix posted here all along" comments below.

Each container has 17 processes... so 17 * 638 = 10,846 total processes. Looking closer I saw that there were 8 "migration" kernel threads per container... which brings the total processes per container up to 25... and 25 * 638 = 15,950 processes. With the host node processes it was up over 16,000 processes and I assume the OpenVZ scheduler didn't want to start any more containers.

Each container uses about 18MB of RAM so that total amount of RAM in use is a little over 10GB. Not too bad considering each container has syslogd, klogd (not really needed), xinetd, sshd, cron, sendmail (two processes) and httpd (1 master process and 8 children) running. Each container was allowed 311MB of RAM (with the split1000 config).

pstree and top header inside of a container

| |-httpd(3447,apache)
| |-httpd(3449,apache)
| |-httpd(3450,apache)
| |-httpd(3452,apache)
| |-httpd(3453,apache)
| |-httpd(3454,apache)
| `-httpd(3457,apache)

top - 13:17:52 up 1 day, 3:02, 0 users, load average: 0.00, 0.00, 0.00
Tasks: 20 total, 1 running, 19 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 319440k total, 18468k used, 300972k free, 0k buffers
Swap: 0k total, 0k used, 0k free, 0k cached

top header from the host node

top - 13:16:12 up 16 days, 23 min, 6 users, load average: 0.22, 0.56, 1.36
Tasks: 16090 total, 1 running, 16089 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.3%sy, 0.0%ni, 98.2%id, 1.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32894804k total, 32730772k used, 164032k free, 1012164k buffers
Swap: 33554424k total, 148k used, 33554276k free, 23391212k cached

I decided to destroy 38 containers leaving just 600 to give it room for some dynamic processes. With 600 containers running there are about 15,140 processes. I made a new sample config for 600 containers and then did an --applyconfig to each one. That made each one have 519MB of RAM available.

Script to destroy unwanted containers

for ((VE=1601; VE<1639; VE++)) ; do vzctl stop $VE ; vzctl destroy $VE ; done

Script to change configs on all containers

for ((VE=1001; VE<1601; VE++)) ; do vzctl set $VE --applyconfig split600 --save ; done

They all have private IPs and I used a few iptables commands on the host node for NAT/SNAT and each container has Internet access, can talk to each other, and the host node can talk to them. Running links text-based web-browser, I can pull any container's default webpage instantly so it doesn't have any trouble at all switching processes.

Script to do NAT

iptables -t nat -A POSTROUTING -s -o eth0 -j SNAT --to {hostnode-IP-address} ;
iptables -t nat -A POSTROUTING -s -o eth0 -j SNAT --to {hostnode-IP-address} ;
iptables -t nat -A POSTROUTING -s -o eth0 -j SNAT --to {hostnode-IP-address} ;


The overall load on the machine at any given time is about 0.32... although at 4AM when the maintenance cron jobs kicked in, the CPUs were only 80% idle.

I'm sure the machine would have to work much harder if each container were actually public and serving up data to the outside world... but overall I thought the results were not too shabby and I was impressed. Of course I wouldn't run that many containers on a single machine in production, but given a slew of rather low bandwidth containers I'm sure several hundred would be fine.

Try that with VMware, VirtualBox or Xen.

I attached a few text documents to this article for anyone who is interested in seeing the pstree -nup listing from the host node... as well as the vzlist -a output... and the sample configs for both the split1000 and split600 sample configs.

pstree-638-containers.txt678.35 KB
ve-split1000.conf-sample930 bytes
ve-split600.conf-sample938 bytes
vzlist-638-containers.txt50.22 KB

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Interested in your start up script.

G'Day Scott

great read we use virtuozzo on our deployments and have similar hardware for our Hardware Nodes so your stats really help direct us in working out some general limits on what we can achieve from each of our servers and then try to convent this into real world stats.

I would really appreciate any advise on scripting our start up sequence for our VZ CT's, we are using the following setup based on VMWare so we can split the servers even more but i feel that we are not getting the best out of this setup, its more flexible but feels limiting at the same time.

In one of our VMWare environment with only 20 CT with 4 Core CPU 16 GB ram we are finding that startups can take up to 40 min and then still take longer to get all the processes up to seed and CT to be responsive.

Im not really sure if its because there are multiple layers of file system ext3 on top of VMWare on top on ZFS that could be causing some delays or if the SAN we are on just has poor performance due to other users also requesting IO but would like to implement a solution where we could stagger CT startups to disperse the load a little.

any advise would be greatly appreciated.


Scott Dowdle's picture

Nothing special in my startup

My startup script is very basic and just does a vzctl start for one container at a time with a sleep in between them of 30 seconds. Starting up 1,000 containers can take a very, very long time... but starting up 20 or so shouldn't take very long.

I have one physical machine that I just set up the other day for a class where each student gets a container... and there are about 22 containers on it... and they start up just fine so far as I've noticed. I'm wondering if your issues are related to you using VMware. Can you try it on a physical machine for comparison?

Revisiting the subject 2 years later: 1890 containers running..

I'm having some problems revisiting your experiment with current hardware.

You mentioned that the limiting factor is disk, what I found is that limiting factor for me seems to be the time needed to start single container,
it looks like having many CPUs doesn't help much, I have a feeling that starting containers serially is a bit faster then starting multiple in parallel.
Also - since it took ~10 hours to start around 2 thousand, rebooting such a machine would be a horrible experience ( especially since I added 'boot=yes' to every vm/container).

Looking at RAM usage, it might be possible to go up to 3 thousand without reducing per-vm memory usage ( I'm using my default template, that among over things means that now I have almost 2 thousand TSM backup clients there up and running ), and maybe a bit higher with replacing sshd with dropbear, and some other similiar tweaks, but then - starting/stopping and generally getting such machine to run, would take multiple days ( and during startup/shutdown of container, whole machine is quite unresponsive ).

Scott Dowdle's picture

More than 1,000 containers?

I started all of my containers one at at time pausing 45-60 seconds between. This seemed to work best for me. Yes, that means that it takes a long time to start up all of the containers. Pausing between each install/startup seems to give the machine a little breathing room so the processes can startup before it goes into another disk I/O intensive template extraction. Also, yeah... you want to turn off auto starting upon boot. Remember, this is an experiment and it will differ some from a production setup.

Try that and see if it helps.

Just out of curiosity, what are your hardware specs?

Hardware: BL465c G7 ( 2x

Hardware: BL465c G7 ( 2x 12-core AMD Opteron 6172 ), 32G RAM.

I used 100G of local disks, and also 200G LUN from EVA 3000 when that wasn't enough.

In the end I used a template that was mostly cp -la'ed as many times as possible ( which resulted in ~9G disk usage for 2.5 thousand containers ).

Sleeping seems to help, but I think it's not the disk I/O that is causing high load/low responsiveness scenario during container startup ( primo - I used pre-created VMs, ie created a thousand, and then tried to start them up, secundo - EVA did not show any exceptionally high IO load )

I also tested starting containers in parallel - and starting two at the same time takes almost exactly as much time as starting two sequentially.

Scott Dowdle's picture

Number of processes

I think when you hit 16,000 processes and go beyond that... you are in the area where the Linux kernel starts getting cranky.

How about the public ip in CT config

Hi,Scott Dowdle

I got a problem in networking config, My host and vm both got public ips in same subnet.

And I use both venet and veth for CT. And I can access My host and CT with their own IP, but the problem when CT access the internet The got the host's public IP not it's own.

btw.I use proxmox ve

best regards.

Scott Dowdle's picture

I don't know network-fu

I'm not that great at more complicated network setups. I'd recommend you email the Proxmox VE mailing list OR get on the Freenode IRC network, join the #openvz channel... and ask our resident network expert ampex.

I'm confident that we can get you fixed up.

Great thoughts

Just wanted to say.. enjoyed this experiment idea alot. If only I had a 600GB vz with 32GigS!

Solaris on commodity hardware

"I've been using Linux longer than Solaris has been around on commodity hardware. When did Sun release Solaris for x86? I started using Linux in January of 1995. Things sure have changed since then."

Solaris x86 was released in 1993. Before that, SunOS was on x86 in 1989.

I ran Solaris 2.5.1 as my desktop on a Micron Pentium back in '96 or '97 and loved it.

Scott Dowdle's picture

Experiment redo 1 year later

I was cleaning out my email yesterday... and let me tell you I had collected hundreds of emails in my inbox and it had been a while. The point though is that I noticed an update to a bug report that Kir (the OpenVZ Project Manager) had filed regarding the error I had run into. The last time I had looked at it was back in April but it appears that Vitaliy Gusev made a suggested work around in July that I had missed. That suggestion was:

Try increase

sysctl kernel.pid_max

to 131072. Default value is 32768.

Sure enough kernel.pid_max was set to 32768 so I increased it to 131072.

Luckily the machine I did the experiment on was still my backup system and the primary was doing fine so I decided to run the experiment again. Of course quite a few things have changed since last year. I was on RHEL 5.2 I think. Now I'm on 5.4... and of course the kernels have changed from what OpenVZ had available last year to the current OpenVZ RHEL5-based kernel (2.6.18-164.2.1.el5.028stab066.7).

But in any event for the follow up test / experiment I my script created 700 containers. It was able to successfully create them and start them all up. With all of that there are 16,894 processes running on the machine.

I'll not bother to create more nor to provide all of the output as I did before because this is such an extreme case that I don't think it is really useful information other than for general knowledge. Maybe when Intel releases their new 48-core CPU and Linux is working on it flawlessly will creating 700 containers on a single machine seem reasonable. :)

Scott Dowdle's picture

But wait, there's more

Kir emailed me and pondered if the amount of disk I had was going to be the limiting factor so I decided keep making containers. How many? Well I had 700 so why not do 300 more and go for 1,000?

It takes a while to make hundreds of containers... but it finally finished and there were 1,000 running (CTID 1001 - 2000).

Number of processes? 24,084

Load? It seems to fluctuate between 2 and 3 when I'm looking. Here's what sar says for the past 24 hours:
Average: CPU %user %nice %system %iowait %steal %idle
Average: all 0.59 0.01 1.01 28.41 0.00 69.97

So, most of the load is iowait and given the number of processes I'm not surprised.

# free -m
total used free shared buffers cached
Mem: 32123 31965 157 0 4533 10341
-/+ buffers/cache: 17089 15033
Swap: 32767 4 32763

# df -h /vz
Filesystem Size Used Avail Use% Mounted on
582G 535G 17G 98% /vz

Looks like I have room for 30ish more containers but given the fact that each container thinks it has 1GB of space with 490MB available, at this point I've really overcommitted the disk. So, I'd say that yeah, the disk is the limiting factor.

I think I'll let the 1,000 containers run for a while just to see if having that many processes on the system makes it less stable or not. I'm not sure I'll be able to go for two weeks like last time but I'll try.

Scott Dowdle's picture

1000 Containers day 2

Creating 1,000 containers takes a while. You have to unpack the .tar.gz, copy a config, start up the container... and then in my script, as previously mentioned, I paused 60 seconds to give the processes time to start up and to give the disk a little break.

Now the bulk of the 1,000 containers have been running for a day. According to sar the iowait dropped from double-digit values to a very small single digit value around 6am. The iowait has been hovering between 1 and 2 ever since... making the average for the 24 hour period 9.79. Unfortunately I just let the container creation process run mostly unmonitored and I think a cron job for updatedb kicked in and I haven't excluded /vz so everything was getting indexed. I don't know if that was the cause for most of the iowait but whatever it was has passed. While the containers were still being created that certainly was the cause for the iowait.

Anyway, the machine's load is in the 0.2x range since the iowait went away with a 97% CPU idle.

I'll see if the smooth sailing continues for the duration. If so, I'll just make one more final comment and if not, I'll document that what happened.

Scott Dowdle's picture

Lack luster closure comment

I ended up running the 1,000 containers for about 1 week and then having to use the machine for a different purpose. I didn't really have anything to report other than the load stayed pretty low, the containers were responsive and I didn't notice anything unusual.

Sorry for the anti-climactic end.

How about the process limit?

How about the process limit? You seemed to say you stopped being able to create more processes at 16k processes. That's quite a suspicious number. What exactly is the limit right now? Is it possible you simply can't create PID's over 16k?

Scott Dowdle's picture

More than 16,000 processes?

When vzctl told me it couldn't start any more containers it was a scheduler error... a special scheduler used by OpenVZ in the 2.6.9 and 2.6.18 kernels. If I remember correctly there was about 16,088 processes so it didn't exactly stop at 16,000... and the host node could run additional processes it was just vzctl refused to start more containers.

Re: More than 16,000 processes?

You can start more than 16000 processes! Just increase sysctl's parameter


Assume that half of pid_max - maximum processes for VEs.

Scott Dowdle's picture

Fix posted here all along

Wow, how did I miss this? Actually, I'm sure I didn't but I must have been busy with other stuff and quickly shrugged it off thinking I didn't have time to do the whole thing over again at the time.

Now we know. :)

Interesting read

Interesting stuff... especially since the CentOS 5 default template has Apache running. How responsive was the server itself? And how about the apache instance inside a container?

How are more real tests performed. KVM (hardware virtualization) for SolidICE is tested against a more real-life situation. You can read about it here:

The situation in which we will probably be using OpenVZ is with different instances for JBoss. As you might know, these suffer from a large memory footprint and behaviour in which memory is not always reclaimed when you redeploy web applications. Other deployed webapps will therefore not suffer from a JBoss restart.

Currently I am on a different assignment, else I would be able to test this soon. This might be a good test to try...

And perhaps you should interpret 'Try that with VMware, VirtualBox, Xen' in a different way. Layer OpenVZ on top of a virtualized CentOS (on VMware or so) and see how it performs. Hardware and OS virtualization can complement each other (not exclude). It is a situation that certainly will be used (that includes me for now).

638? is it a lot or not?

638? is it a lot or not? I don't fully understand why it failed to create more than 638 containers.

Scott Dowdle's picture

Me neither

To me 638 containers is a substantial number. The error given really didn't tell me a lot so I'm not sure. Perhaps I should ask the OpenVZ folks and see what they say.

OLPC Linux VServer

Interesting, Ivan Kristic formerly of the OLPC project once observed:

"The interesting thing about this, by the way is you know people are terrified of how are you going to do virtualization with 466MHZ CPU - turns out with Linux Vserver that the overhead you pay is 32 K per task struct, but there is 0% measurable CPU overhead with up to 65,000 Virtual Machines running."


I am currently trying to port the OpenWRT rootfs as a container. This might help.


error number 12 = ENOMEM.

You ran out of memory, not processes. If each container was using 18MB of RAM, then 638 containers used only 11GB of RAM. There's quite a large amount of RAM unaccounted for in the containers as more than 31GB of the 32GB of ram on your host is used...

Scott Dowdle's picture

One error or another?

According to top I had plenty of memory available. Maybe it was related to memory that is usable by the scheduler? I don't know. I haven't found a good reference for the error. You have one?

ENOMEM just means something

ENOMEM just means something in the kernel ran out of memory.

Most likely you ran into memory fragmentation on the kernel stack. The kernel stack on x86-64 requires two continuous in physical memory pages and when memory fragments it's often difficult to get that after some time. You can check that by looking at /proc/buddyinfo: if column 6 is all 0 you ran out of order 1 memory blocks.

Nice... and now compare it

Nice... and now compare it to linux-vserver, please.

Scott Dowdle's picture

Linux-VServer rocks too!

While I'd like to be able to do the same experiment with Linux-VServer, I'm not as familiar with it as I would like to be... and I don't currently have a server of the same class with Linux-VServer installed on it.

I certainly do welcome contributed articles to this site... hint, hint.

Hmm... you are certainly NOT

Hmm... you are certainly NOT comparing apples to apples with the comment:
"Try that with VMware, VirtualBox or Xen."
I think you might want to take your OpenVZ against the originator of the Containers idea... Solaris.

See what happens when you take ext3 or ext4 or reiser or whatever, up against ZFS. I'm not sure how well the FSS works in OpenVZ, so I'll have to take a look at it myself.

Also, xVM - Solaris' implementation of Xen is quite a bit more powerful than the conventional Xen, however it's not aimed at the Container model at all.

Xen, VmWare, and VirtualBox all allow you to run operating systems other than your host O/S and kernel. Eg you can run Windows or some other unix side by side the linux host. Try that with OpenVZ.


The ISPs of the world have been providing low cost operating systems level containers to their costumers for a while.

FreeBSD 4 (March 2000) opened up the possibilities of containers to the service providers. The technology was solid, but it did not come easy to use tools and it was technologically a bit limited compared to other solutions we see today.

"The FreeBSD jail mechanism is an implementation of operating system-level virtualization that allows administrators to partition a FreeBSD-based computer system into several independent mini-systems called jails.":

"The jail utility appeared in FreeBSD 4.0.":

"4.0-RELEASE appeared in March 2000":" target="_another2">

Scott Dowdle's picture

And then?

Yes, I have heard of FreeBSD jails too... but like you said, they aren't quite as feature complete as containers... and a bit difficult to work with. I wasn't trying to say that OpenVZ was "first". Heck, chroot would be first.

Scott Dowdle's picture

Different Fruits?

Yes, that comment ("Try that with VMware, VirtualBox or Xen") was made in jest. I certainly understand the differences between machine / hardware virtualization and OS virtualization. One point is that some people use the machine / hardware virtualization products for Linux on Linux hosting where OS virtualization clearly kicks its butt... although I do acknowledge there are some Linux on Linux virtualization scenarios that OS virtualization is not well suited for... but there are a lot of scenarios where it is well suited.

Anyway, I doubt that Solaris had Zones before SWsoft released Virtuozzo (from which OpenVZ comes). Virtuozzo was originally released in 2001 although I don't know the month. Perhaps I'm wrong about that but I haven't found any evidence of that.

Regarding filesystems, Linux/OpenVZ does not have anything like ZFS and the closest thing you can get to it would be using a combination of RAID and LVM. I use RAID and LVM but not LVM snapshots... so yeah, OpenVZ takes up lots more disk space. Virtuozzo uses vzfs which does pool files... so it saves disk space on common files between containers. OpenVZ does not have that feature. That would be nice to have in situations like my experiment where I create a lot of containers that are all the same.

I have looked into Solaris Zones but I've never quite understood them. Take for example this tutorial that I happen to have run across yesterday:

Devzones: Simple Use-and-throw Developer Environments

Granted it is about using Zones on Nexenta that is based on the OpenSolaris kernel... but as far as I know, it doesn't vary much from Zones in Solaris proper.

I need an article that better explains Zones because so far I've read several and I just don't get them. They sort of sound normal until I see all of these special commands run by users to create a zone... and then they destroy it? If you could point me in the right direction for a more comprehensive article, I'll check it out.

OpenVZ (and Linux-VServer) containers are just easier to understand... grouped processes into a container which is fuctionally a virtual machine with it's own resource parameters, distro, filesystem, root / users, etc... something you usually want to hang around because you created it for a reason.

A feature I believe that is missing in Solaris Zones that is avialable with OpenVZ is offline and live migration.

Solaris Zones and Containers

Well, you are right about the age bit - it was in 2004 that Zones in Solaris took off - with the Sol 10 release.

Solaris Zones are able to be migrated, but I've never used this feature.
Not sure I'll want to with ZFS anyway. I'll just attach another array and add to the pool.

Of course I can always export the ZFS slice and then import it on another machine. Using AVS is quite a nice method of dealing with this - it's all included in OpenSolaris anyway.

Don't get me wrong, I like Linux - I'm using Linuxmint for my desktop at work right now. I work at a hospital though, and since I've had problems with ext3, reiser3, and of course ext2... well I'm a bit gunshy of using linux in mission critical production where my name is on the chopping block.

Having actually built a zpool out of 5 of those WD 500G USB attached drives - and having it hold 2TB data in a live backup situation - for over 3 years... and lose a drive, have it replaced, and all work - without rebooting, totally sold me on Solaris/ZFS.

For desktop, I use Linux, it's well suited. For server, I use Solaris, it's more powerful. This is just an opinion... but I'm likely to use the best tool for the job when allowed, and whatever the vendor specifies when I have to. Case in point, I mix Windows 2k3 environments running Tomcat with iSCSI mounts to Solaris ZFS exports - where they sit as drive D: - this is where I load Tomcat and the webapp. But because the company expects windows 2k3 with tomcat to do their upgrades, I have to live with that... doesn't stop me from using all the benefits of ZFS though.

Anyway, I used to be a die-hard Linux nut, avoiding all windows, all anything but Linux for about 5 years. Debian and Gentoo were my only flavors... it's funny, that kind of hermetic seal doesn't really do much to benefit a person either. Windows is a good system in its own way, and it's really nice to be able to subvert it when all the windows guys say it can't be done.

Well, if you look at the bottom of that link, you'll find tons of sun docs for zones. Head on over to for good unix info. are a good place to be. One thing to be said is that sun documentation is amazing and very detailed - but you WILL need it. If you don't like reading, you won't like Solaris.

well, I've written enough.

Scott Dowdle's picture

I'm not anti anything... but

I attended a Sun sponsored event a couple of months ago and the message was clear... having been repeated over and over... Sun is an OpenSource company... and they want to be like Red Hat. Now they didn't exactly say that last part but the first part was like a mantra. Only problem is that they aren't open enough... at least for me. I'll give them props for FINALLY releasing Java under the GPL and all that they have contributed to the FOSS world. I don't know if GPL'ing ZFS would help them out or not... I'm not a stock holder and I don't care about their financials... but I know that is their number one concern.

To me Sun, regardless of the quality their software and hardware, is just another example of the bygone Unix days. They are like SGI... trying desperately to make it in a market place that seems to be lapping Linux up. They have been releasing their technology in order to please the FOSS mindshare but like with SGI, it hasn't helped much. I don't have any data to back this up but I believe Sun's marketshare has been shrinking for some time now. How important is that? For the short term not too important but for the long term... yeah, just ask SGI... or any other commercial Unix/BSD vendor. Why is Linux winning? I'm not sure but I think a lot of it has to do with the openness.

That isn't to say that I prefer inferior technology. I would love to have ZFS as a part of Linux but Sun has made sure that can't happen. Ok, they created ZFS, that is their right. I respect that. Linux doesn't have anything as good as ZFS at the moment, I concur... but what it does have is good enough for me. I haven't had any ext3 crashes in recent memory. The hardware RAID I have on my (important) servers has worked well for several years now... I keep good backups... and LVM is there but I haven't taken much advantage of that yet. I guess I'm just not that mature/skilled when it comes to storage technology. I do look forward to the maturization of the various up and coming filesystems that seem to have a chance at rivalling ZFS.

I'm pretty loyal to Linux but that doesn't mean I'm going to bury my head in the sand. If I reach a point where Linux isn't cutting it for my needs I will look elsewhere.

I've been using Linux longer than Solaris has been around on commodity hardware. When did Sun release Solaris for x86? I started using Linux in January of 1995. Things sure have changed since then.

I wish Sun well in its fight to stay alive but I won't be too surprised if it doesn't make it. Of course people have been predicting the end of Apple for like a decade or two but they seem to have pulled themselves out of the fire. Maybe Sun can too. Perhaps I'm even overstating it... maybe Sun isn't even in the fire to begin with.

I do somewhat resent someone telling me that I should use something else because it is better. Where have I heard that before? There are various ways to measure better. As you said, from a desktop users standpoint, there is a pretty strong case to be made for Linux. I think so on the server too... but I don't try to tell anyone they should use Linux (or a particular Linux distribution) because it is better. Much of "better" is about preference... and we all have our own.


Reading your statement about Sun been able to keep alive on 2008 and now bought by Oracle just make me laugh..

will keep reading this page, maybe i can make a buck in the future with your forecast about IT companies


Heh, I'm not telling you to use Solaris at all...

But I was defending my favorite OS. I really don't think that's at all wrong. And, yeah, I've been using Linux for a while myself - since around mid-year 96, when Win95 crashed on me, I gave up on M$ and tied out something called Debian 1.1... those were interesting times - I was in college working part time on the DEC Alphas, trying out Linux for the first time, trying to keep a dying HPUX machine alive, heh, the good ol days.

Anyway, if you've taken offence to something i said, well I guess that's the way it goes. If you think Sun is going away, well maybe it is, maybe it isn't. I do think they are restructuring their products to better serve a marketplace - take a look at the unified 7000 series of storage arrays.

Oh, and btw, sun didn't say Linux couldn't have ZFS... hell, FreeBSD, Mac OS/X and OpenBSD all use ZFS now. Don't tell me Sun is stopping Linux. Linux has ZFS in their Fuse methodology too - it's just crappy slow there. The problem is the licensing. Sun doesn't like GPL-V2 or GPL-V1. They do like GPL-V3 and have said they will re-release their software under that license as it's much more compatible with their CDDL license method.

I do get tired of the "Oh, it's a big company, and they want to make money... therefore they are evil. Screw them" mantra... Sun is a big company, yes, but are they flexible? yes? Tell your CPU open-source? Oh, wait, you use x86... so, nope, your CPU technology is closed source. How odd is that... does open-source only apply to software? Or are open standards nice to apply to hardware too? If you are wondering why I even bring this up... take a look at the SPARC platform. It's an open-source or open standard CPU platform from which a few vendors build their server. Recall that Fujitsu makes CPUs as well - oddly enough they are based on the open standard called SPARC - and hey, they have put their improvements back into the standard. Look at how much more flexible the SPARC is vs. the x86. x86 is ubiquitous, common, and most of all cheap. It's hard to get inexpensive SPARC stuff... I don't even have one myself. I, too, use closed source hardware... but I can't wait to move to SPARC when I get the chance.

Anyway, I'm not trying to rant, just trying to show that there is always more than one side to the argument. If we end up in an all-Linux world... what kind of choice is that? We'll just have the one-size fits all crap again - and then there will be room for the pendulum to swing back to MS - or some other proprietary system. I'm all for variety - and not just more Linux distros... Variety means incompatibility sometimes, trade offs others, but overall variety increases competition, and competition breeds excellence.

well, have a good day!

Scott Dowdle's picture

Jack in the box?


Thank you for your comments. They are great. I don't really argue or get offended... I discuss things... reasonably. You made some good points regarding SPARC.

I've read about their multi-core, 8-thread design and it is pretty darn awesome. They even include encryption features in the hardware. I wonder what software it takes to take advantage of that?

I'm not really anti-profit. Heck, I'm a Red Hat fan, right?

I'd really like to see Sun release ZFS under some license (or dual-licensed) that is compatible with the GPL-2 that Linux is licensed under. It was made fairly clear though, that Sun wasn't interested in that. Of course perhaps they'll change their mind.

I don't think Linux will last forever. Something has to take its place but what I'm not sure. Look at some of the stuff AST did after Minix... and the additions he has made to Minix 3. And of course there are a large number of OSes out there, mostly in niche areas.

I completely agree that competition is good.

ZFS etc.

I dunno about Linux not lasting forever, though of course eventually the sun will go nova...

Open source software has kind of a different life cycle from proprietary stuff. Consider GCC -- one of the earliest C compilers. It's still thriving, though I would bet that very few lines of code survive from version 1.0. OSS tends to adapt: I've been running Linux on my desktop since 1.02 and can hardly believe how it's exploded in 14 years. (Yeah, I ran it on a 386 with 4M of memory; Netscape was statically linked with the Motif libraries. When it started to load, get a cup of coffee. Hell, make a pot of coffee...)

Speaking of Sun going nova -- as to ZFS, I predict that either some of the ext4 geeks will read over the Solaris code and say, "Hey, I can do this a better way" -- which true geeks always say -- and then proceed to do it a better way, or Scott McNeally will get tired of being poked by IBM guys at Palo Alto cocktail parties who say, "Look, we gave 'em JFS, what are you gonna do?", and then poked by SGI guys who say, "Look, we gave 'em XFS, what are you gonna do?"

That said, though, I'll be interested to see what eventually happens with pthreads. Solaris has always been the champion of low-overhead thread scheduling and multiprocessing; Linux 2.6 is extremely good in that respect and has made incredible progress, but it's still not up to Solaris' level. If Sun would turn some of its programming talent in that direction, as IBM did with NUMA ...

I'm still not sold on the

I'm still not sold on the idea that Sun, because they wrote a nice goodie, should just give it all away. I'm glad they do, but I see the greedy behavior of the "I want this, I want that" Linux crowd to be distasteful. Linux seems to really really be slowing down in the innovation arena. Apache, Samba, Openldap, Glassfish, Tomcat, Java, MySQL, PostgreSQL, etc. All those are great opensource projects - not Linux. the fact they run on Linux is really nice... But where's the really cool neato breakthrough on Linux? They seem to not be happening? I recall the day when I finally got usb working - some 5 years ago, man was I excited and just unreal proud to be a linux user. What seems to be happening to me is that Linux is attracting a lot of the undesirable windows crowd, people who complain immediately about this or that without reading the man pages, or understanding the nature of the word "Google". So, here Linux is, another welfare nation, depending upon the handouts of others... Sun invests some $2B in R&D yearly for their projects... and Linux wants all their stuff for free. We all want I suppose. I just happen to look a Linux as a Ford Mustang, and Solaris as a Mercedes SL600. Both are powerful and fast, it's just that Solaris is way more luxurious - well written man pages, good documentation at - every possible way for a person to figure it out for themselves. Declaring Sun dead is like saying transport by rail is going away. Good luck - I think you're in for a surprise in the coming years as to Sun's ability to change markets and grow.

In my opinion, we are in for a marked change in cpu usage - sparc is so much more powerful, so much more efficient than x86, it's about time AMD took on more of the Sparc methodology and produced a chip which could run at least 4 threads per core.

Well, opinions and butt holes, we all have them... hopefully we are not spewing too much waste.

Scott Dowdle's picture

Off topic? That's ok


Sun has already given away ZFS to a certain extent... as several OSes that are compatible with the license they chose have already adopted it. What is annoying is Sun's mixed messages and constant direction changing. I guess that is supposed to be good and means that they are nimble and can change dynamically as the market changes... depends on ones perspective. Sun has decided to release some things under the GPL... take Java for example... although it took them a long time to do it.

While you see the Linux community as freeloaders let me remind you that there are a few commercial Linux companies that seem to be doing well even though they live (some better than others) the ethics of the GPL. Red Hat isn't as freedom oriented as the Free Software Foundation but they seem solidly rooted in the idea that what is good for the customer is good for them. To me Red Hat has a pure freedom model (notice the difference between free price and freedom) that is working for them.

Sun seems to kind of blow with the wind. Again I am thankful for and acknowledge their vast contributions to the FOSS community over the years. Check out the Open Source pony tail video for a satirical expression of Sun's business model. Having attended a Sun sponsored event recently I can tell you first hand that they are basically trying to morph themselves into a FOSS company... like they want to be Red Hat or something.

Linux isn't innovating but other projects are? I guess you just don't get it. Linux is a kernel. Distribution makers gather up the software. Various projects make the software. Sun understands this model very well because with OpenSolaris they are trying to adopt the Linux distro model... and are moving away from their Sun-only software model to everything that can possibly offer them and their customers more value. I think they have been choosing a lot of GPL'ed software and as a consumer of third-party software I would imagine they prefer GPL'ed stuff over less free software. If that isn't the case with the OpenSolaris distro makers, I'm sure it definitely is with OpenSolaris customers. The GPL protects the freedom of the user / customer.

So far as Linux as a kernel innovating... yes it is happening at a breakneck pace... in every direction. Linux is like a liquid and it flows in every direction that nature / gravity takes it. It is being used in embedded devices some of which are quite small both physically and with regards to resources... all the way up to the mainframe. There are a lot of extra flavors available to those that want them: hard real-time, soft real-time, advanced messaging, HA & HP clustering, grid computing, Linux native virtualization (KVM and in the not too distant future containers), security platforms (SELinux and AppArmor). Linux is moving in so many directions all at the same time that many see it as directionless and chaotic. The reason this is happening is because people can do with it as they wish.

Kernel development, after a certain level of maturity, is all about just maintenance and adding drivers for hardware. I think Linux totally kicks Solaris' butt with regards to hardware support. Not in the Sun hardware support area but all others.

The plain fact is that both Linux, Solaris, and third-party projects continue to innovate and borrow from others. If I had to judge I'd say that Solaris is borrowing more from Linux lately (not in the kernel space but in the distro space) than the other way around. I'd prefer to see the welfare used by individuals than by corporations. :) I'm not really complaining about Sun borrowing from Linux and third-party projects... it is about time. Most Sun users ended up manually borrowing anyway... and Sun has just decided that they can do more of that borrowing themselves rather than leaving it up to the customer.

Sun is very much a mono culture. I don't think you'll see them develop a filesystem that competes with ZFS. Linux? Well, they aren't too bothered by the existence of ZFS and the fact that they can't have it. Linux has a ton of filesystems and continues to develop more... and the internal competition is good.

As you know Sun has bought up a few projects (Star Division [StarOffice /] and MySQL for example) and I think the general consensus, after the initial injection of cash, the software projects have been somewhat mismanaged. There have been a number of articles claiming that as a project is not very healthy and that it is Sun's management style that has lead to that. I don't know enough about it to make a judgement.

So far as analogies go, I don't have a good one for you. Linux is so complex and amorphous and Sun is quite unique too... that any analogy I could come up with would only address one or two properties and fall apart with any serious scrutiny.

Why not have it all?

Why is the assumption that full virtualization and containers are mutually exclusive? There is at least one solution which provides the best of both worlds:

Proxmox VE provides the ability to run, manage, and migrate both fully virtualized virtualmachines via KVM, and containers via openVZ.

Scott Dowdle's picture

Best tool for the job?


Thanks for the comment. I'm very familiar with Proxmox VE. I gave a presentation on it at a BozemanLUG meeting a few months back and I have advocated it for use where I work... and it seems to be catching on.

I hope to have an in-depth article about the latest Proxmox VE release published in the not-too-distant future written by a BillingsLUG member (kaptk2). He has been working on it for a while but with the holiday season making us all rest and enjoy ourselves, it might be delayed until January. We'll see. In the mean time, enjoy the article he wrote some time ago about the initial Proxmox VE release.

I really like the Proxmox VE approach. I like the fact that KVM is in the mainline kernel and look forward to the day when Linux Native Containers is fully implemented in the mainline kernel.

Also if you check out the interview I did on The Linux Link Tech Show you notice I mention Proxmox VE (about 37 minutes into it) and speak quite favorably of it.

I also appreciate that the Proxmox VE folks give back a lot to the community. First of all Proxmox VE is GPLed.. and I've seen PVE developers be very active on the OpenVZ mailing lists... submitting patches to a few of the OpenVZ tools. They also wrote the VM backup utility vzdump.

Great Test Case

This is a real interesting article.

How would this work on a number of older machines running Openmosix or some other?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.