Discovering SmartOS


As I’ve alluded to previously, I’ve been exploring Joyent’s SmartOS lately. I’d like to talk a little bit more about it. It seems to me that a lot of my peers haven’t yet heard about it, or don’t really understand it much at all. The documentation isn’t really there yet, so I think that it is a little hard to get the right first impression if you’re coming into it with no Solaris background.

That’s right, I said Solaris. SmartOS is not Solaris, but it shares a common heritage with Solaris. SmartOS is based on the Illumos kernel, which is based on the OpenSolaris project (which no longer exists). Oracle has since closed the Solaris source, and most of the best engineers have found themselves reuniting under the Illumos banner. Many of them are under the employ of Joyent and Nexenta.

Illumos provides a common kernel and some of the fundamental system tools. Distributions build on top of Illumos and offer a unique branded experience. There are quite a few out there, but I’ll just mention a few that I’ve experimented with and my impressions of where they fit.

  • OpenIndiana is going to be much more familiar to those that came from a Solaris or OpenSolaris background. Of the distributions that I’ve tried so far, this is the distro that I would want to run on a desktop or even a laptop.
  • Nexenta CP is the community distro provided by Nexenta. If your background is in Linux, especially Debian or Ubuntu, this is going to be a very familiar environment. Quite a large portion of the Debian user experience has apparently been ported over to sit on top of the Illumos kernel. Debian package management is here, too, but with a twist: on Nexenta CP, ZFS filesystem-level snapshots are leveraged when you install packages, giving you an easy way to roll back if you need to. This distribution is undergoing some major changes right now and becoming what is known as “Illumian” (think “Illumos” + “Debian”).
  • SmartOS is the community distro provided by Joyent. It seems to mostly be a subset of their commercial SmartDataCenter product, minus the secret sauce that gives you a full-blown multi-datacenter cloud infrastructure. Of the three Illumos distributions, I think this is the one that is the most innovative. You’re not going to use this for your desktop. But you might use this to run your cloud applications.

Tonight, I want to talk to you a little more about SmartOS. It’s been sort of a poorly kept secret infatuation of mine lately, one that is blossoming into a full-blown romance.

Like any good romance story, this one had an unlikely beginning. I’m a system administrator. I’ve been one for about 18 years now. And while I’ve managed quite a variety of platforms in my time, ranging from Windows to AIX to Solaris, a little HP-UX, more recently FreeBSD, for the last 14 years or so it would be accurate to say that I’ve been a Linux guy at heart.

Linux has been my bread and butter for many years. Like any good IT guy back in the 1990’s, I snuck Linux into environments where it wasn’t welcome. I helped to get IBM’s Linux Users Group off the ground, which was actually a rather dangerous move during those years. I was also part of a rebellious group of geeks that thought it would be a good idea to have an enterprise-class Linux distribution that everyone could afford (read: free beer), so CentOS was born (though for awhile, it was cAosity, and I’m really glad the name got changed). Linux has been an amazing and wild ride for me for all of these years.

Solaris was always around, but always so damned ugly to me. I mean, let’s be honest. The command line tools were mostly not as good as GNU, and some really great common commands just weren’t there at all. Coming from Linux or *BSD, everything was just… weird. I never really got the hang of it, and I didn’t really want to. At every opportunity, I was replacing Solaris machine with Linux or OpenBSD. Quite a few times, I was installing OpenBSD right on the sparc hardware, which helped to keep up the illusion with management that we were still a Sun shop.

I’m not going to lie and say that the Illumos distros are going to be familiar or friendly to someone who only knows Linux. There is going to be a learning curve. Accept that now, and get over it. You’ll be glad you did. I’m still climbing that curve, and I’m being richly rewarded for it. The fact of the matter is, there are a number of killer features in SmartOS and other Illumos distros that you just don’t have in Linux. OK, so there is NWAM but everybody admits that pretty much sucks (unless you’re using a laptop, in which case it might be tolerable).

SmartOS especially is fighting an uphill battle when it comes to attracting new users. They are doing things very differently from everybody else, and like children pushing peas around on the plate, we’re all sort of looking at it fearfully. I’ve heard some less forward looking sysadmins make complaints about why it doesn’t have feature parity with VMware (it’s not VMware, you big dummy!) or why in the world it boots from PXE or USB or a CDROM instead of from hard disk (actually this model works really well once you “get it”). The userland tools aren’t exactly Solaris, aren’t exactly GNU. The package management isn’t exactly NetBSD. And some of the virtualization tools are straight out of the future.

SmartOS has almost no footprint on your hard disks. Your hard disks are for zones, where your virtual machines are stored. If you really want to, you can virtualize Linux or Windows guests within KVM containers. Oh yeah, SmartOS has KVM. Did I mention that? Well, it does. The other type of zone you can provision is a native zone, called a “Joyent” zone. If you’re coming from a Linux background, the closest thing you’ve got to this is probably OpenVZ as a point of comparison. But this is so much better than that.

Ideally, you would want to set up a PXE boot server for your SmartOS hosts. I plan on writing an article on that at a later date. SmartOS is delivered as a CDROM ISO image, or you can also boot it from a USB thumb drive. My old frankenbox isn’t booting from USB happily, so I’m booting from CDROM for now. An upgrade to PXE booting is planned, and I’ll take you with me through that.

The first time you boot SmartOS on a new machine, it checks out the disks and notices that the ZFS filesystem that it’s looking for doesn’t exist. All of your zones are kept in this ZFS filesytem (that sounds so redundant when I say that redundantly). So SmartOS will give you a list of disks that it sees, asks you which ones should be included in ZFS, and then it creates the filesystem for you. You’ll also need to answer the prompts for your TCP/IP configuration before rebooting. When it comes back up, it’s ready to go. You can log in at the console as “root” with the password of “root” (gotta love it… I hope for some improvement there). Hopefully the very first thing you do will be to run passwd and change root’s password.

Now it gets fun. Let’s spin up a new virtual machine, shall we?

Your hostname will look like a string of hex digits and you might be saying “WTF” to yourself. Relax. Everything in SmartOS is treated like a UUID (Universally Unique Identifier). Get comfortable with that now, because you’re going to be seeing more of it.

To provision a new virtual machine, first you’re going to want to install a template. We can do that pretty easily. First thing we need to do is install a dataset, which is sort of like a virtual machine template, that we’ll clone to make our guest zone from. Let’s see what datasets are available:

[root@00-1b-fc-58-ce-97 ~]# dsadm avail
UUID                                 OS      PUBLISHED  URN                    
467ca742-4873-11e1-80ea-37290b38d2eb smartos 2012-02-14 sdc:sdc:smartos64:1.5.3
7ecb80f6-4872-11e1-badb-3f567348a4b1 smartos 2012-02-14 sdc:sdc:smartos:1.5.3  
1796eb3a-48d3-11e1-94db-3ba91709fad9 smartos 2012-01-27 sdc:sdc:riak:1.5.5     
86112bde-43c4-11e1-84df-8f7fd850d78d linux   2012-01-25 sdc:sdc:centos6:0.1.1  
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd smartos 2011-12-19 sdc:sdc:smartos64:1.5.1
39c4372c-2a50-11e1-a7fa-4bebdf5eb6e6 smartos 2011-12-19 sdc:sdc:smartos:1.5.1  
362f57e8-24d0-11e1-bbe6-e74758fd20c6 smartos 2011-12-12 sdc:sdc:riak:1.5.3     
b4afcce4-1f8d-11e1-b4e4-fb3ae78b3768 smartos 2011-12-05 sdc:sdc:riak:1.5.2     
e05dbcac-1d44-11e1-b8ab-bf1bc04c2d65 smartos 2011-12-05 sdc:sdc:smartosplus64:3.0.7
fcc5996a-1d34-11e1-899e-7bd98b87947a smartos 2011-12-05 sdc:sdc:smartosplus:3.0.7
0ac8200c-13b5-11e1-a042-7b2948257b3b linux   2011-11-21 sdc:sdc:centos57:1.1.6 
fa07b5b6-118e-11e1-913c-3f246c23837c linux   2011-11-19 sdc:sdc:fedora14:0.1.0 
a77907fc-123b-11e1-86d8-2bb3c7c6aaf2 linux   2011-11-18 sdc:sdc:centos6:0.1.0  
56108678-1183-11e1-83c3-ff3185a5b47f linux   2011-11-18 sdc:sdc:ubuntu10.04:0.1.0
e483afce-10b2-11e1-86bc-ff468add832f linux   2011-11-17 sdc:sdc:debian603:0.1.0
70ce41ae-0cbe-11e1-8c2c-3f2bb88d2a6a smartos 2011-11-14 sdc:sdc:smartosplus64:3.0.6
aa7aeea4-0cbc-11e1-b3cb-cf1c7e291b56 smartos 2011-11-14 sdc:sdc:smartosplus:3.0.6
5fef6eda-05f2-11e1-90fc-13dac5e4a347 smartos 2011-11-03 sdc:sdc:percona:1.2.2  
d91f80a6-03fe-11e1-8f84-df589c77d57b smartos 2011-11-01 sdc:sdc:percona:1.2.1  
475a1026-00a4-11e1-a2df-a71425f9c46b smartos 2011-10-27 sdc:sdc:nodejs:1.3.0   
8f61457a-f40a-11e0-87bd-dbd638565dbb smartos 2011-10-11 sdc:sdc:riak:1.5.1     
a9380908-ea0e-11e0-aeee-4ba794c83c33 smartos 2011-09-28 sdc:sdc:percona:1.0.7  
df3589dc-df9a-11e0-a3a3-07ceee3e7d54 smartos 2011-09-15 sdc:sdc:smartosplus64:3.0.4
aded640a-df98-11e0-b050-1f55ff3ddfa7 smartos 2011-09-15 sdc:sdc:smartosplus:3.0.4
9199134c-dd79-11e0-8b74-1b3601ba6206 smartos 2011-09-12 sdc:sdc:riak:1.4.1     
3fcf35d2-dd79-11e0-bdcd-b3c7ac8aeea6 smartos 2011-09-12 sdc:sdc:mysql:1.4.1    
141194fa-dd77-11e0-8539-27dd8d8264b8 smartos 2011-09-12 sdc:sdc:smartos64:1.4.7
f8ea0bb8-dd75-11e0-87c3-af5352ad3bd6 smartos 2011-09-12 sdc:sdc:smartos:1.4.7  
a1175b2e-db23-11e0-9cef-d7ba3111e77b smartos 2011-09-09 sdc:sdc:smartos:1.4.4  
47e6af92-daf0-11e0-ac11-473ca1173ab0 smartos 2011-09-09 sdc:sdc:smartos:1.3.18 
80716640-cfec-11e0-8eb8-bf491eb79cd3 smartos 2011-08-28 sdc:sdc:smartos:1.4.3  
be1d48ec-c4ca-11e0-9935-eb833f38dc43 smartos 2011-08-12 sdc:sdc:smartos64:1.4.2
47edd64e-c071-11e0-98c4-5bc02808198f smartos 2011-08-10 sdc:sdc:smartos:1.4.2  
bb6d5a10-c330-11e0-8f18-9fbfcd26660b smartos 2011-08-10 sdc:sdc:smartos:1.3.17 
47e7cf16-bf5d-11e0-9c9d-77106185df1a smartos 2011-08-05 sdc:sdc:smartos:1.3.16 
7abad50e-b865-11e0-8462-730bcfdde020 smartos 2011-07-27 sdc:sdc:smartos64:1.4.1
70856d1a-b860-11e0-9bbf-bf2a7494dd82 smartos 2011-07-27 sdc:sdc:smartos:1.4.1  
184c9b38-ad3d-11e0-bad6-1b7240aaa5fc smartos 2011-07-13 sdc:sdc:smartos:1.3.15 
63ce06d8-7ae7-11e0-b0df-1fcf8f45c5d5 smartos 2011-05-10 sdc:sdc:smartos:1.3.13 
41da9c2e-7175-11e0-bb9f-536983f41cd8 smartos 2011-04-28 sdc:sdc:nodejs:1.1.4   
7456f2b0-67ac-11e0-b5ec-832e6cf079d5 smartos 2011-04-15 sdc:sdc:nodejs:1.1.3   
febaa412-6417-11e0-bc56-535d219f2590 smartos 2011-04-11 sdc:sdc:smartos:1.3.12 
[root@00-1b-fc-58-ce-97 ~]#

OK, so we can see there are a bunch of SmartOS branded datasets available, each providing different capabilities or different versions. There are also some Linux branded zones, which run inside of a KVM container. The first column gives us the UUID of the dataset, which is what we’ll need to actually import the dataset. Let’s try grabbing the latest 64 bit SmartOS dataset.

[root@00-1b-fc-58-ce-97 ~]# dsadm import f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd doesnt exist. continuing with install
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd successfully installed
dataset f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd successfully imported
[root@00-1b-fc-58-ce-97 ~]#

That will take a minute or two to run, maybe longer depending on your connectivity. It’s actually going out to Joyent’s cloud and making an API call to fetch the dataset.

Let’s see what datasets are available locally now.

[root@00-1b-fc-58-ce-97 ~]# dsadm list
UUID                                 OS      PUBLISHED  URN                    
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd smartos 2011-12-19 sdc:sdc:smartos64:1.5.1
[root@00-1b-fc-58-ce-97 ~]#

Excellent. Now we need to create a new zone from this dataset. You’re going to want to

man vmadm

and make some notes on the options. We’re basically going to put together a plaintext file in JSON format that contains the options for our new zone.

{
        "alias": "honeybadger",
        "brand": "joyent",
        "dataset_uuid": "f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd",
        "dns_domain": "example.com",
        "quota": "10",
        "nics": [
                {
                        "nic_tag": "admin",
                        "ip": "192.168.0.5",
                        "netmask": "255.255.255.0",
                        "gateway": "192.168.0.1"
                }
        ]
}

Create a file like the example above. You can adjust the IP addresses to suit your own network topology, as well as the DNS domain name. Save it to

/tmp/honeybadger

. Note the leading whitespaces in my example above are tabs, and your leading whitespace must also be in the form of tabs or this won’t parse properly.

Creating your new zone couldn’t be easier. This might take a minute or so to run so be patient:

[root@00-1b-fc-58-ce-97 ~]# vmadm create -f /tmp/honeybadger 
Successfully created 0720ce96-f0dd-40c7-b01c-36356bb2e915
[root@00-1b-fc-58-ce-97 ~]#

Yes, it’s that easy. Don’t believe me? Ping the IP address that you put in

/tmp/honeybadger

:

[root@00-1b-fc-58-ce-97 ~]# ping 192.168.0.5
192.168.0.5 is alive
[root@00-1b-fc-58-ce-97 ~]#

Hey, let’s try logging into the console. You can use the UUID for your zone that the

vmadm create

command gave you as output.

[root@00-1b-fc-58-ce-97 ~]# zlogin 0720ce96-f0dd-40c7-b01c-36356bb2e915
[Connected to zone '0720ce96-f0dd-40c7-b01c-36356bb2e915' pts/5]
[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]#

Outstanding! You might wonder what packages are installed:

[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]# pkgin list
zlib-1.2.3           General purpose data compression library
sudo-1.7.7           Allow others to run commands as root
sqlite3-3.7.7.1      SQL Database Engine in a C Library
readline-6.2         GNU library that can recall and edit previous input
postfix-2.8.5        Postfix SMTP server and tools
pkgin-0.5.1          Apt / yum like tool for managing pkgsrc binary packages
pkg_install-info-4.5nb3 Standalone GNU info file installation utility
pkg_install-20110805 Package management and administration tools for pkgsrc
pkg_alternatives-1.5 Generic wrappers for programs with similar interfaces
pdksh-5.2.14nb5      Free clone of the AT&T Korn shell
pcre-8.13            Perl Compatible Regular Expressions library
patch-2.5.9          Patch files using diff output
openssl-0.9.8qnb3    Secure Socket Layer and cryptographic library
ncurses-5.9          CRT screen handling and optimization package
nbsed-20040821nb1    NetBSD-current's sed(1)
nawk-20050424nb3     Brian Kernighan's pattern-directed scanning and processing language
mozilla-rootcerts-1.0.20110902 Root CA certificates from the Mozilla Project
man-1.6e             Man page viewer and utilities
libiconv-1.14nb2     Character set conversion library
libarchive-2.8.4     Library to read/create different archive formats
less-444             Pager similar to more and pg
joyent-tools-1.0     Various tools relevant to Joyent SmartOS and products
install-sh-20100824  install script compatible with the BSD install program
gtar-base-1.26       The GNU tape archiver with remote magnetic tape support
gsed-4.2.1nb1        GNU implementation of sed, the POSIX stream editor
groff-1.20.1nb5      GNU roff text processing suite
grep-2.5.3nb1        GNU grep
gettext-lib-0.18.1.1 Internationalized Message Handling Library (libintl)
genbfpw-0            Simple tool to generate Blowfish crypts of passwords
gcc-runtime-4.6.1    GNU Compiler Collection 4.6 Runtime libs
gawk-3.1.8nb1        GNU awk
findutils-4.2.33nb1  The GNU find, xargs and locate utilities
diffutils-2.8.1nb5   GNU diff utilities - find the differences between files
db4-4.8.30           Berkeley DB version 4 from Oracle
cyrus-sasl-2.1.25    Simple Authentication and Security Layer
coreutils-6.12nb4    GNU basic file, shell and text manipulation utilities
changepass-1.3.3     Simple tool to set user passwords non-interactively
bzip2-1.0.6          Block-sorting file compressor
bootstrap-mk-files-20090807nb2 *.mk files for the bootstrap bmake utility
bmake-20110606       Portable (autoconf) version of NetBSD 'make' utility
[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]#

The

pkgin

tool is how you’re going to install new packages inside of your zone. If you run it without any arguments, it’ll give you a nice summary of its options. I’ll be honest, a lot of the packages now are pretty old and some of the dependencies are questionable. This community could really use some packaging help. I’m hoping to adopt a few packages myself as I get to know the ins and outs of SmartOS better.

If you want to be able to ssh into your new zone, you’ll want to paste your OpenSSH public key into

~/.ssh/authorized_keys

and then

chmod 600

the same filename. You cannot ssh in as root with a password.

I’m already in excess of 2,200 words now so I should probably shut my trap at this point and allow you to digest this much. We’ll get into more depth and learn some cool tricks in upcoming installments of this blog. I hope this was a useful introduction to building your first zone, even if it’s a very high level overview. I’m still learning this myself, and will gladly accept corrections and suggestions for other articles about SmartOS.

Creative Commons License
Discovering SmartOS by Magnus Hedemark is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Based on a work at opusmagnus.wordpress.com.

Advertisements

43 comments

  1. Thanks for writing this. SmartOS has been an enigma to me (even just looking across from the OpenIndiana camp), so it’s great you’re shedding some light on it.

    1. Thanks. I’ve felt a lot of the same pain. It turns out, SmartOS is actually pretty easy to use and do neat things with. The documentation hasn’t really caught up with the capabilities, so I’m hoping to shed some more light on this.

      I’m definitely planning on writing more articles soon. Some (like hosting Linux and other OS’s inside of KVM containers) will have to wait until I can upgrade from my old AMD box to a newer Intel machine, since KVM on SmartOS only supports Intel.

      Definitely planning on writing more about provisioning, automation, package management, zone management, ZFS, how daemons work in SmartOS, and more.

  2. Thanks for the nice article, I have been reluctant to spend time learning SmartOS because of my irritation with Solaris in general but the technology makes it very tempting.

    After reading this post it makes me want to get a decent Intel box and try KVM linux/windows guests on SmartOS and see how it compares.

    1. I definitely understand your reluctance to get into anything that descended from Solaris. I’ve been there, myself.

      I’ve been working with a very old white box with an AMD Athlon 64 X2 6000+, 4GB of RAM, and 2x 500GB Western Digital “green” SATA2 disks. Hardly a “decent box”. If you’re just looking for something to learn on, I don’t think it has to be too terribly modern (Core 2 Duo maybe?).

  3. I just brought up my 1st SmartOS system, thanks to this. Built a ‘zone’, but, I ran into this when attempting to ssh into it using a userid I created on it:
    Feb 15 16:50:40 a6bb5a1a-c8ec-419d-9369-f003fc0729de.wm.edu sshd[4083]: [ID 800047 auth.error] error: Can’t convert to UTF-8: Cannot convert the local codeset strings to UTF-8

    I googled that & see the issue (if I was running Solaris 11, which I kinda am). Did you run into this by any chance?

  4. Well, my inability to ssh in was not due to the above message, it appears that sshd is config’d to only let you in with a key in your authorized_keys file. Which I had, but, I ‘hand-made’ the account & home directory as root, and, root owned the .ssh directory (which I had chmod 0700 on). Lesson learned – chmod 0600 authorized_keys and make sure the user owns that and the .ssh directory. LOL.
    I also learned that the folks at #joyent at irs.freenode.net are very helpful – even though it was a simple fix, they offered up assistance. Even Anthony – the author here popped in and said hi to me there.

    BTW – we’re a Solaris 10 shop, looking at 11. We use zones extensively, in fact almost every service runs in a zone, our Global zones are used only for admin’ing the system. The Windows team uses VMware and also has some Linux VM running there for our Networking folks. I resisted VMware, as I always thought Solaris with zones, zfs and dtrace were just too good to pass up. SmartOS looks to be the best of both worlds.

    SmartOS is looking good. Now onto to playing with it some more.

    1. Outstanding! Glad you got some help. Yes, I’m finding the community to be very helpful. And Joyent is taking their responsibilities to this community very seriously, I’m finding.

      I have a lot of ESXi at work and honestly I wish we were using SmartOS or SmartDataCenter instead.

  5. Thanks so much for this, really helps clarify some things for me and shows how superb SmartOS really is. I think as the community writes more articles like this the adoption rate will go through the roof!

  6. Just a note – instead of creating the zone config file in /tmp, I created a directory /zones/Myfiles to place the files in, that way they persist across reboots of the global zone (which I find handy).

    I used this for almost 2 days (I’m an expert now – LOL). I’ve been able to create a SmartOS system in which I installed & ran jasig cas (a central authentication server – runs in tomcat), ocsinventory – a web/php/mysql based Inventory server, and, a debian kvm system. Following this guide and using the docs at wiki.smartos.org it was pretty easy to do. Granted, none of these are production servers, but, it’s all fairly easy to do – heck – I’ve also compiled a ton of stuff that we use locally on it (Solaris servers), just to make sure that would go ok. Even copied over some binaries compiled on a Solaris 10 system – just to make sure they would run ok – they did (as long as the libs are in place).

    Have to say I’m not surprised, but, I am impressed!

    Now I want to update my homebrew server at home so I can run this on it (wife is gonna love that, a weekend spent at the keyboard).

      1. Great idea. I’m getting way out of control with this now – been tossing anything I can think of at it – the only stuble I’ve had was attempting to install Sun Directory Server 5.2 in a joyent ‘zone’ – the setup scripts requires libXm.so – some old Motif library.

  7. hey this all looks very “smart”. maybe your kind self or one of your readers can help me out…. I have funds in a Nigerian bank account… haha, just joking, but seriously. pxeboot? you would set up an image on another server to serve your smartos box. to me it seems a bit much to run a second server (as I want smartos to be my main server), would it not be a better to have an internal (external if you wish) usb thumbdrive as the boot drive. probably, but maybe there is something that i might not be understanding about pxeboot so I would appreciate any quick thoughts on this until your pxeboot post is, um, posted.

    finally, how cool would a freebsd zone be?

    1. In your case, a thumb drive or a CD might be more appropriate. PXE makes a lot of sense for larger installations with multiple servers.

      If I had some modern Intel hardware, I’d love to set up guests for a number of different platforms so I could work on pkgsrc packaging. I’m working from an oooold AMD box, though, so no KVM for me unfortunately.

  8. Reblogged this on Nick Zivkovic's Old Blog and commented:
    Introductory and informative blog posts like this one are needed to take SmartOS (and other Illumos-based tech) beyond the murky, mysterious realm of kernel engineering, and into the intensely pragmatic realm of common system administration.

  9. Great write up, would have saved me 1-2 days.

    SmartOS is very very cool, I almost used it in production, as it would let me use all 4 of my drives for data (I don’t like the idea burning two drives for a mirrored OS with illumian or openindiana).

    After I got a few bugs solved and integrated (with the help of the user/chat groups like (https://www.illumos.org/issues/1439 ) I hit a stupid bios issue on my chosen HW

    Unfortunately I had an issue the new brain dead BIOS/EFI (IBM xSeries server) limited read speeds USB thumb drives load the 275MB images at a mere 0.75MB/sec. Of course after boot I can get 26MB/sec, but the boot took 10+ minutes the customer wouldn’t tolerate that. I didint; want to PXE boot and I just don’t have the cycles to try to make a EFI grub/boot loader. Note this isn’t really a SmartOS issue but a vendor/HW issue on other servers SmartOS worked great.

    Jon

    1. I played with SmartOS it took me a while to get my head around it. I kept wanting to install it on a hard drive (mostly due to my choice of HW switching from Oracle/Sun to IBM which clouded my judgement for several weeks), not then I realized (actually accepted the design choice) that my 4 drive 1 RU servers could have 4x2TB all for data and VM’s – wow I didn’t have to burn a drive bay (actually two I always mirror) for an OS.

      So I am a convert, my only issue is that some servers like UEFI SystemX have pig slow 0.75MB/sec legacy boots. This does hurt with a traditional OS but think a) load 275MB image b) execute image in RAM.

      Obviously not the fault of the Joyent folks, but a native UEFI grub boot sure would be nice. My sun x86 and Supetmicro servers boot SmartOS in 30 sec, but the IBM SystemX stuff takes 6.5 minutes to boot.

      As to OmniOS, yes I will try it illumos based πŸ™‚ but wonder about if I will like it more than oi_151a2 (load 151a build a new BE delete original BE). Perhaps the ability to roll my own distro’s might convince me to use OmniOS, but seriously I think Joyent’s SmartOS approach coulpled with sple full OS ( kerp tour zones and VMs) method of upgrade is too cool and pratical to not switch over too.

      Okay 1) I really really like SmartOS 2) OpenIndiana gives me a warm Solaris felling and can be upgraded if need be to bleeding edge illumos – plus I can install OI from a live CD in about 5 min.

      So the only reason I will look at OmniOS is to use and play with compiling my own distros. Why – it’s damned difficult (some might say imposdible) with OI, thus it is easier for mist to wait a year between OI releases.

      Wait …. Support now here’s the Million $ question …. Can it work …. Yes OmniIT could pull it off …..

      Wouldn’t be nice if OmniIT, gave back e.g. 1) propose their build system for illumian, SmartOS, and Openindiana and 2) make templates for each of them to prove viability. Then OpenIndiana, SmartIS, OmniOS and Illumian – would all be common easily supported variants of the illumos core.

      This should actually help adoption of ImniOS and IMHO a future business model of providing support. SmartOS keeps their cloud based management secret sauce, Openindiana becomes a build able distro, and OmniIT can sell support services to all! How by tracking pkg/build manifests and supporting known “gold” versions of any illumos based OS If there is an issue “off standard” downgrade/upgrade to an approved manifest rev document the issue and open a support ticket.

      (*) Full disclosure – I am in no way financially affiliated with any company mentioned other than being both an Oracle and IBM reseller.

      1. Sorry about all the typos above – big fingers small iPhone – killing time by the pool on holiday

  10. Thank you for this smart article. I liked SmartOS from the first time I heard about it. Currently it’s all about learning curve getting deep into it.

  11. Magnus, thanks a lot for great writeup — it really made clear the very ideas standing behind SmartOS. Are you still planning to write some howtos about installing VMs with KVM usage? Like Linux for example?
    I have couple concerns regarding security — there are no firewall settings you set for SmartOS, so basically you need to rely completely on properly securing zones and VMs, right?
    I was also wondering about networking — does zones and VMs _without_ dedicated IP have access to the outside world? Is it being handled be some kind of built-in NAT? Or not at all and you need to set it up somehow differently?
    How about setting MAC address for you NIC manually (I’m talking about zone/VM)? Is it possible or not?

    1. Filip,

      Thanks for commenting. I’m only using SmartOS on one home server now. My home is all IPv6, minus this one server and its guests. The pain of IPv6 on SmartOS presently is a constraint on my enthusiasm for further adoption of it. I’ve got some OmniOS and Debian in the home network now, as well.

      The scope of the blog article was pretty narrow. I did not get into zone security or other issues at all.

      My zones are using VNICs and have their own IP addresses on the same network as everything else, and have access to the outside world.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s