#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

Kyuubi MetaStore Upgrade HowTo
============================

This document describes how to upgrade the schema of a Derby backed
Kyuubi MetaStore instance from one release version of Kyuubi to another
release version of Kyuubi. For example, by following the steps listed
below it is possible to upgrade a Kyuubi 1.6.0 MetaStore schema to a
Kyuubi 1.7.0 MetaStore schema. Before attempting this project we
strongly recommend that you read through all of the steps in this
document and familiarize yourself with the required tools.

MetaStore Upgrade Steps
=======================

1) Shutdown your MetaStore instance and restrict access to the
   MetaStore's Derby database. It is very important that no one else
   accesses or modifies the contents of database while you are
   performing the schema upgrade.

2) Create a backup of your Derby metastore database. This will allow
   you to revert any changes made during the upgrade process if
   something goes wrong. The easiest way of accomplishing this task is
   by creating a copy of the directory containing your Derby database.

3) Dump your MetaStore database schema to a file using Derby's dblook utility:

   % dblook -d <derby_db_url> -z "APP" > my-schema-x.y.z.derby.sql

   Note that "APP" is Derby's default schema for user-created catalog
   objects.


4) The schema upgrade scripts assume that the schema you are upgrading
   closely matches the official schema for your particular version of
   Kyuubi. The files in this directory with names like
   "metadata-store-schema-x.y.z.derby.sql" contain dumps of the official schemas
   corresponding to each of the released versions of Kyuubi. You can
   determine differences between your schema and the official schema
   by comparing the contents of the official dump with the schema dump
   you created in the previous step. Note that due to a bug in Derby
   the order in which the DDL statements appear is non-deterministic,
   so simply diffing the two dumps is unlikely to result in useable
   results. A simple workaround for this problem is to compare sorted
   versions of the two schema dumps, e.g:

   % sort metadata-store-schema-x.y.z.derby.sql > metadata-store-schema-x.y.z.derby.sql.sorted
   % sort my-schema-x.y.z.derby.sql   > my-schema-x.y.z.derby.sql.sorted
   % diff metadata-store-schema-x.y.z.derby.sql.sorted my-schema-x.y.z.derby.sql.sorted

   Some differences are acceptable and will not interfere
   with the upgrade process, but others need to be resolved manually
   or the upgrade scripts will fail to complete.

5) You are now ready to run the schema upgrade scripts. If you are
   upgrading from Kyuubi 1.6.0 to Kyuubi 1.7.0 you need to run the
   upgrade-1.6.0-to-1.7.0.derby.sql script, but if you are upgrading
   from 1.6.0 to 1.8.0 you will need to run the 1.6.0 to 1.7.0 upgrade
   script followed by the 1.7.0 to 1.8.0 upgrade script.

NOTE: You may need to install the Derby 'ij' utility.
      Look here for installation instructions:
      http://db.apache.org/derby/docs/10.4/getstart/

   % ij
   ij version 10.4
   ij> CONNECT 'jdbc:derby:/Users/bob/hive/metastore_db';
   ij> RUN 'upgrade-1.6.0-to-1.7.0.derby.sql';
   ij> RUN 'upgrade-1.7.0-to-1.8.0.derby.sql';
   ij> quit;

   These scripts should run to completion without any errors. If you
   do encounter errors you need to analyze the cause and attempt to
   trace it back to one of the preceding steps.

6) The final step of the upgrade process is validating your freshly
   upgraded schema against the official schema for your particular
   version of Kyuubi. This is accomplished by repeating steps (3) and
   (4), but this time comparing against the official version of the
   upgraded schema, e.g. if you upgraded the schema to Kyuubi 1.7.0 then
   you will want to compare your schema dump against the contents of
   metadata-store-schema-1.7.0.derby.sql