SPM and schema virtualization

I received a lot of feedback and a fair number of good questions on my session on SQL Plan Management for the Virtual Technology Summit. Some of the questions deserved additional research and a more detailed answer than I could give in the live Q&A and I hope I can add most if not all of these issues to this blog eventually. A lot of you have also asked for the slides and i uploaded the pdf here in case you have trouble pulling them from the event site. Here is the first one: Continue reading

OTN MENA tour final stop: Dubai, UAE

After a very relaxed day I followed the others virtually on their travel to Dubai. All but Osama were too tired to go out and opted for a quit night and early sleep. So after a quick run that left my soaked in sweat I met Osama for some catching up since I have not seen him since UKOUG Tech 13 back in december.

Dubai Marina at nightDubai is a very exciting destination. Two million people from all over the world are living in a rather compact space that has erupted from the desert in the past few decades. It has become the business hub of the golf region with many major banks and services companies occupying a stunning set of skyscrapers, all of which are overshadowed by the majestic Burj Khalifa, the world’s tallest building at 830 meters.

The event location was chosen wisely by Vishnu Vinnakota of prosec in the sizzling marina district at a very nice hotel which featured the largest projector screen I have had the honor of presenting on. There were no complaints of a too small font for some of the code examples this time.

We started with the proven Exadata performance expert panel which we have done at most of the stops before and all that practice made perfect, we even spiced things up a bit by letting someone else present “their” slides.

OTN MENA tour Dubai teamIt was the third time I saw Ed present his introduction to Big Data on this tour but he is so much fun to watch and the presentation flows so nicely that it still did not get boring for me.

Mike talked about IO performance and started with a review of how physical disc drives work and why they are limited to a maximum of 200-300 IOPS each. He then showed how this number can be increased by using RAID technology, compared this to modern flash based devices and showed ways how to benchmark and monitor the IO system. Great basic content that should be known by all DBAs.

speaking in dubaiAfter a lunch that had me go for seconds three times, I was scheduled to talk about flashback data archives. But I also had requests from the attendees to talk about SQL plan management and 12c new features. All three are full 45 to 60 minute presentations but I managed to get to the gist of each of them in about 15 minutes and was able to deliver all three topics.

Osama discussed high availability options for fusion middleware from self-written scripts to full blown cluster and provided many tricks to manage those things in the stack above the database that we dbas so often neglect and forget about.

I was excited to meet fellow ACE Director Joel Perez for the first time and he did not disappoint when he presented about how to upgrade from 11g single instance to 12c RAC with minimal downtime. In addition to this specific project there was also a lot of information about backup, restore and replication.

The final words once again belonged to Jim who wrapped it up in style while the rest of us once again admired his eloquence in speaking and presenting. Unfortunately, he had to cut his presentation short because we (the others) overran a bit on time which is a shame because I was very much looking forward to his examples on histograms. I saw him prepare the slides on the plane to Cairo and thought it was quite hilarious. I guess this will now premiere with another audience at another conference (and I will not give away what it is).

Overall, the tour was a huge success, especially given the fact that this was the first of its kind and these were all the first time we had done anything in these cities and did not really know what to expect. A big thank you to Tariq and Syed who masterminded this tour, the local event organizers who did their very best to provide the forum for us to present and reach out to the local users and of course to the Oracle Technology Network and ACE program for being the sponsor and enabling us do to this.

I very much hope that we can establish the tour as a solid event in the region and keep improving it year by year. I know that I and all of the other speakers would be proud and more than happy to be back again.

OTN MENA tour Stop 2: Saudi Arabia

As I am writing this I am just slowing down after very intense 36 hours in Riyadh, still in awe and slowly processing all the impressions I made here. We arrived at King Khalid airport at 9pm on Wednesday night but hat to test both our own and our generous hosts’ patience as we waited 4 hours in the immigration line. I was expecting that with a pre-approved visa (that process alone was another huge effort) this would be rather quick. Ironically, they are in the midst of upgrading their biometric database and are not able to run at full capacity right now.

OTN MENA tour 2014 RiyadhWith the late arrival we had to skip all dinner plans and were happy to fall into our hotel beds and sleep at least 4 hours until it was time again to get up for the event which was held in the conference center of a very nice hotel. And Esolutions, the local sponsor, together with the ARABOUG went out of their way and made a huge effort in organizing this. They had signage, a very nice stage, incredible arabic lunch and they used the intermissions to award to distinguished guests and raffle off a number of very nice prices among the more than 100 Oracle DBAs and professionals that showed up for this first of it’s kind event in Saudi Arabia. One of the VIPs that was awarded an award shared how he started his IT career as a junior DBA with one of Mike’s books. There was even a professional photographer and a TV crew to document the day.

OTN MENA tour Riyadh - Jim speakingAll presentations were spot-on and well received by a crowd that was hungry to expand their knowledge. HA and RAC seemed to be the topics of the biggest interest and we managed to cover a lot of ground, starting with the Exadata Tuning Expert Panel led by Tariq, my RAC management session, Mike, Inam and Jim talking about performance tuning and Syed and Ed rounding it all out with sessions on security and big data. In addition, I also enjoyed very much to chat with a great number of attendees and hopefully answered a number of very specific questions. One of the biggest pieces of advice I hope I could give is that if they enjoyed the event it is really up to the local oracle community to organize themselves and meet for informal roundtable-type events and share their knowledge rather than hoping and waiting for an OTN tour stop every year. With Syed and Inam Ullah Bukhari they already have at least two great local speakers there and I am sure there are more.

OTN MENA tour awardEverybody was very polite and I am sure I shook hands, said thanks and had my picture taken with every single attendee at least twice during the day. After many heartly good-byes it was time for us to go on a brief sightseeing tour of Riyadh. While it may have been brief, we still managed to see so very much of this wonderful city that is by no means a common tourist destination due to their very strict visa policies. Our hosts took us to the national museum Which showcased the history of the region from the creation of the universe to today and included impressive models of the holy mosques in Mecca and Medinah including the drape that covers the Kaaba. The museum was structured very concisely and was so huge that it was really a shame that we had to rush through it.

OTN MENA tour RiyadhThis was followed by a walk around of a nice park and what looked like the rebuild of old Riyadh with small alleys, open areas surrounded by palm trees and nicely decorated houses. We then had the chance to visit Kingdom Centre, Saudi Arabia’s highest building (until the Kingdom Tower in Jeddah is finished which should become the worlds tallest building), and zipped up the the 100th floor where you can enjoy magnificent views from a bridge that spans the two arches that give the tower a very distinct and unique appearance.

The night ended with a very generous dinner by the local sponsors and while the food satisfied the hunger, I wished the conversations would have gone on for another few days. But there was barely enough time for a very short two hour nap before at least Jim and I had to head out to the airport again which gave us the opportunity to see the sunrise. Tired but still excited, my mind was so preoccupied that I left my cell phone in the taxi but was able to get the driver’s number from the hotel and he turned around to return it. Just calling my own number did not even cross my mind until a few hours later.

The event in Jeddah was only scheduled for half a day, so I would not speak there and proceeded directly to Dubai. I was actually thankful for one day of rest on the beach. But I still followed the Jeddah event over the live stream and pictures on social media and it looks like that one was another success. Now I am waiting for the others including Joel and Osama to arrive in Dubai for the final event of the tour.

OTN MENA Tour Stop 1: Tunisia

I landed in Tunis after a rather uneventful flight via Istanbul on Sunday afternoon and met up with Tariq Farooq, Mike Ault and Jim Czuprynski at the hotel where we enjoyed magnificent views of the bays surrounding the suburb of La Marsa. Our local guide and ARABOUG’s business development manager in this area Hussem who showed us around and explained the origins and workings of the ARABOUG to us. This user group spans the whole region of the middle east and northern africa and represents about 300.000 professionals.

OTN MENA tour BizerteThe actual events were being hosted by two universities in cities other than the capital of Tunis. On Monday, we were picked up in the wee hours and driven to Bizerte in the north. We set up camp in the auditorium of their technical university and quickly began our presentations with an introduction of the speakers, OTN and the ARABOUG. In addition to local Oracle professionals there were many interested students in the audience, about 150 people in total. What surprised us all was the amount of women in the audience, among 60-70% of the students were skilled and highly enthusiastic females. They were actually very surprised when we we told them that at our technical universities 10% is already considered very high.

OTN MENA tour speakersPresentations were very well received and for me it was an extra pleasure to listen to the other speakers and their expertise. Mike started with a keynote overview of his 24 years of working with Oracle technology, Tariq gave a great overview of Exadata, its features and special things to consider while Jim captured the audience with tidbits of technical details, general advice and motivational pieces.

Mohamed Houri connected with us remotely from France and did a presentation on adaptive cursor sharing in french, the native language spoken in Tunisia along with arabic.

OTN MENA tour BejaEd Roske completed our small group on Tuesday when he came to Beja directly from the airport and despite the long-haul flying and jetlag just jumped up on stage and gave a great and lively definition, explanation and introduction to what Big Data really is and what type of questions can better be answered by non-relational database systems.

We enjoyed great hospitality by the local event organizers who went out of their way to shuttle us around between the different venues, taking in sites from beautiful beaches to gigantic ruins of the roman empire, feeding us and providing a base for many great conversations.

As much as we enjoyed Tunisua, the crew is also very much looking forward to the next two stops in Saudi Arabia, where we will meet Syed and finally Dubai where Joel Perez and Osama Mustafa will already be waiting for us.

PS: I am writing this at the airport in Cairo and my phone still has not seen an update for their DST change and is hence showing the wrong time. It took five smart IT guys and a lot of googling to determine that our next flight will be 2:30 instead of 1:30 as shown by our calendars and a number of websites.

egypt creates timezone chaos by re-introducing DST with one week notice

I have a presentation that I give from time to time with the title “Tackling Time Troubles – how Oracle handles datetime data” and in less than 10 days I will embark on a tour where I will deliver this presentation. One of the issues I talk about is how different nations handle daylight saving time changes and how much of a pain this can be. I did not anticipate this one though. As part of the MENA tour I will be flying with egypt air from Tunis through Cairo to Riyadh. Today, I received a very strange message from egyptair. IMG_1253

This has to do with the egyptians deciding on very short notice (1 week) that they want to re-introduce daylight saving time. And because they are already behind the date when everybody else in Europe advances their clocks, they chose _tomorrow_ (May 16th) to do so. This is potentially bad for a couple of reasons:

  • it is very confusing if you need to coordinate meetings or phone calls across countries. The rest of the world would really like to know what time it is wherever you live
  • computers that calculate these things have to be reprogrammed by people like me. This usually takes a while. My iPhone is giving me the wrong time. Oracle has a MOS note (1670704.1) about this but the latest timezone file patch does not yet include this change.
  • schedules (for trains and flights) get all messed up during the switches. In the switch forward, trains will appear to arrive an hour late and on the switch back, they would really need two trains during the night (because there is one hour that “happens twice”. Or in the case of the Deutsche Bahn they simply stop all their trains in a station for one hour during the night. Either way it is madness.
  • Flights to egypt from other countries will appear to arrive an hour later than what is written on the schedule

Egyptair is taking care of the latter one by simply delaying all departures by one hour:

The customers who issued their tickets before 23:59 Thursday 15th of May 2014:
The departure time will be one hour later than the time stated on the tickets.

While I do appreciate that I get to keep my comfy 1:45 of connection time, I also see this has a huge potential for chaos. If I look up the flight schedule on their website, both arrival and departure time are updated. But sites like flighttracker still show the old times, even for tomorrow. Also, if everybody in egypt simply advances their schedules by one hour, the effect of the DST change are nullified.

On top of all this, they seem to also have the idea of “suspending” DST during the islamic month of Ramadan. This would be between June 28th and July 27th in 2014. The IANA _guesses_ that these switches would actually happen on the 29th of June and 29th of July at midnight but no official word seems to exist.

Governments of this world: Would you please decide things with more than 7 days of notice? Preferrably 6 months or more so that we can comfortably and properly implement your ideas into the already very confusing hacks that keep track of all this timezone weirdness.

quick tip: build debian package from sqldeveloper rpm

Oracle has made the installation of sqldeveloper easy on distributions that use rpm by providing a nice package. But we are using ubuntu for our development workstations at the office and thus need a debian .deb package. I do not want to use the ‘alien’ workaround and we already build or rebuild some of our packages for our own repository anyway. This can easily be built with the help of fpm like this after downloading the Linux rpm from OTN:

fpm --no-auto-depends -d 'oracle-jdk-7' -s rpm -t deb -m "portrix Systems GmbH " /var/tmp/sqldeveloper-4.0.2.15.21-1.noarch.rpm

Upload this to your local repository and you can then install this with apt-get or simply show the details like this.

portrix@vm-04:~$ sudo apt-cache show sqldeveloper
Package: sqldeveloper
Version: 4.0.2.15.21-1
Architecture: all
Maintainer: portrix Systems GmbH 
Installed-Size: 305622
Depends: oracle-jdk-7
Provides: sqldeveloper
Filename: pool/main/sqldeveloper_4.0.2.15.21-1_all.deb
Size: 246647402
MD5sum: 1a7127bde4521e0a5bc7160ee3f891a0
SHA1: 55f12afe5075bcbd9223f61fe8f183fd0570c295
SHA256: aeb07cede84298c7c8a9bf8caf0c42ccf560840a857415be81bb4189c8939ba2
Section: extra
Priority: Priority
Homepage: http://example.com/no-uri-given
Description: Oracle SQL Developer is a new, free graphical tool that enhances productivity
             and simplifies database development tasks. With SQL Developer, you can browse database objects, 
             run SQL statements and SQL scripts, and edit and debug PL/SQL statements. 
             You can also run any number of provided reports, as well as create and save your own.
License: Oracle

RACAttack hands-on workshop at OUGF in Finland

A bunch of RACAttack Ninjas are once again gathering to deliver an installfest workshop as part of the OUGF Harmony conference in Helsinki. Attendees can bring their own laptops and with some instructions and the help of experiences RAC DBAs will setup a 12c RAC Cluster on a red stack (Oracle Linux and Oracle VM VirtualBox). Or simply drop by to work together with someone else or network and engage in some geek talk.

RACattack OUGFRAC Attack isn’t just about RAC. During the workshop you learn how to setup one of the most complex infrastructures on your laptop. You learn how to configure and clone virtual machines, how to create and setup internal and external virtual networks, create different storage types, setup a local DNS server and other challenging components.

After this workshop you have enough knowledge to setup a test/sandbox environment for virtually any product.

You can find us on Thursday from 12:00 to 16:00 in Lappish Kota 1. In addition there will also be a RAC Attack hacking session in a traditional finnish sauna later that day, you do not want to miss this! The title of this hot mini-session: “Why size does matter but bigger is not always better”. And if all goes well we will also be able to give you a sponsored souvenir (more on that later).

To participate in the workshop, participants need to bring their own laptop. Please check the minimum hardware requirements and download at least the grid and database installers ahead of time. You are also encouraged to bring your favourite Humppa song!

resize linux root disk in Oracle VM

I just needed to grow my root fs on an Oracle Linux guest VM running in OVM 3.2 and I thought I’d share the process with you. A lot of the steps are taken from a similar blog on doing a similar thing on vmware.

To start with, I have a linux vm which started as a clone of one of the Linux VM templates that Oracle provides. It comes with a 20G disk which is partitioned into two pieces. One for /boot and the other one is lvm physical volume that holds the rest of the system including the root partition. Continue reading

changing archivelog mode on 11g and 12c RAC

I hate being wrong. But it does keep happening. Last week I sat through a RAC installation (12c GI with 11gR2 database) with a client and one of the steps involved enabling archivelog mode for one of the databases. They shut down all instances, then started one of them in mount mode only.

srvctl stop database -d RAC
srvctl start instance -d RAC -i RAC1 -o mount

Just as they logged on to sqlplus and started typing “ALTER DATABASE ARCHIVELOG;” I proclaimed that this was not going to work unless they first changed CLUSTER_DATABASE in the spfile on restarted that instance. Blank looks. An enter key was pressed. To my surprise the database came back with:

SQL> ALTER DATABASE ARCHIVELOG;

Database altered.

Apparently this has changed after 10gR2 and it is not longer neccessary to fiddle with CLUSTER_DATABASE while switching archivelog mode. For years and dozens of installations I have done this extra step believing this was the way it has to be done. But not any more. I re-learned something.

revisiting Solaris 11 Automatic Installer without DHCP

A while ago I blogged about my experience with the new automatic installer in Solaris 11 and my special setup in which I refused to use a DHCP service because DHCP is simply something we do not use in our datacenter. That particular blog post has been one of my most popular (by visits) so this must be something that other admins experience aswell.

One of the issues I came across was that while you can easily specify IP, gateway and installation server on the commandline without DHCP, the install will break at a point where it needs to install some packages from ‘pkg.oracle.com’ but could not resolve that name due to a lack of DNS servers being set up. I described a rather tedious workaround in that post but came up with something a little better today. Continue reading

Cisco AnyConnect Static IPv6

We are using Cisco AnyConnect to provide VPN for our users (which we authenticate via RADIUS). Recently we needed to provide some users with static IP addresses.

For IPv4 this was easy since the ASA we are using supports RADIUS attribute 8 (Framed-IP-Address). For IPv6 there is an equivalent RADIUS attribute 168 (Framed-IPv6-Address) defined in RFC 6911. Unfortunately the ASA doesn’t support that attribute.

But thanks to the Cisco support we found out that the ASA (starting with version 9.0(1)) does support RFC 3162 (Cisco Bug ID CSCtr65342). What RFC 3162 provides are RADIUS attribute 96 (Framed-Interface-Id) and 97 (Framed-IPv6-Prefix). With these two you can easily provide static IPv6 addresses to your AnyConnect users.

To assign a user the address 2001:db8::42/64 you just set the following attributes in your RADIUS server:

Framed-IPv6-Prefix = 2001:db8:0:0::/64
Framed-Interface-Id = 0:0:0:42

For setting a static IPv6 address for local users one can use vpn-framed-ipv6-address which is documented in the Release Notes and the Command Reference.

firewall port for ZFS Appliance with Oracle VM Storageconnect

I just stumbled across this and could not find it anywhere else on the net. I set up a ZFS Appliance with Oracle VM and their storageconnect plugin according to the documentation pdf (which are pretty easy step-by-step instructions) but in this case the OVM Server and the ZFS Appliance were not in the same network and access is denied by default in the firewall between those nets. So trying to register the appliance as a FC Storage led to this error that just tells me that the connection timed out. Continue reading