There are a lot of different levels of virtualisation, but we'll focus on server virtualisation to start with. You take the software running on a physical server, OS and all, and run it instead in a virtual machine. One virtue of this (and perhaps not the most important) is that it lets you run multiple virtual machines on a single physical server.
Now, though, think about what has not changed. You have avoided tinkering with the original operating system (perhaps Windows or Linux) and all the software running on it, including the application. This is one of the appeals of virtualisation: you have a way of getting more out of your assets, without the risk and inevitable delay (and expense) of migrating from one application or one operating system to another.
But there is a price.
That multitude of operating systems, and perhaps the applications running on them, are part of the problem. They still have to be managed. The OS and the hypervisor are doing some of the same jobs, and you're paying for it. Although new chip design has boosted performance there is still a lot of unnecessary code being executed. In one virtual machine it may not seem much, but if you have a thousand, then it might add up.
What is an operating system? An OS has two main _raisons d'etre_:
To manage and share the hardware resources amongst the programs using them
To provide a set of services (APIs) that application programs need
This was pretty straight forward on the mainframes. There was one collection of hardware to be managed and shared. There was one operating system, and it supported many application programs, used in turn by thousands of users. If there was virtualisation, it was under the covers.
So what's different now?
Compared to the size of the total workload in a big organisation, the size of a server is small and you tend to have lots of them. You have a pool of resource to manage.
We already have applications. They expect to run on Windows or Linux.
We don't want virtualisation. It's just a step on the way. Using virtualisation is a huge, blazing sign that there are more steps on the path. It's a way of handling old applications (and the old operating systems we are obliged to use to run them.
One operating system that controls all the hardware resources
One set of services used by all the software running on it
Many of the applications running in a large company require a number of servers. Virtualisation isn't needed to squeeze several of these onto one server.
What makes virtualisation so useful here is that it deals in images of complete "virtual machines" saved as files. They can be moved as easily as files and launched on any server running VM ESX or the relevant hypervisor.