Monday, November 10, 2014

Oracle 12c: Invisible column


In 12c, Oracle has introduced new feature called Invisible column. It is used to protect data from application users.

- Columns can be made invisible in the CREATE TABLE statement or subsequently using an ALTER TABLE statement. By default columns are visible.
- Invisible columns can be made visible again using an ALTER TABLE statement.
- Any generic queries on table (like SELECT or DESCRIBE) will not show invisible columns.
- The invisible columns are not included in the column order for the table.
- Invisible columns are not seen unless specified explicitly in the SELECT list.

Example:

1) Create table with invisible column
SQL> create table test (a number, b number invisible, c number);
Table created

- Normal describle or select query will not show hidden column

SQL> desc test
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 A                                                  NUMBER
 C                                                  NUMBER

SQL> select * from test;
no rows selected

- We can insert into hidden column without specifying it.

SQL> insert into test values(1,2,3);
insert into test values(1,2,3)
            *
ERROR at line 1:
ORA-00913: too many values


SQL> insert into test values(1,2);

1 row created.

SQL> select * from test;

         A          C                                                        
---------- ----------                                                        
         1          2                    

- By default NULL value will be inserted for hidden column. We can insert value to hidden column explicitly by specifying the column.

SQL> select a,b,c from test;

         A          B          C                                              
---------- ---------- ----------                                              
         1                     2                                              


SQL> alter table test modify b visible;

Table altered.

- Since invisible changes the column id, the order it stored in database gets changed.

SQL> select * from test;

         A          C          B                                              
---------- ---------- ----------                                              
         1          2                                                        

SQL> insert into test values (1,2,3);

1 row created.

SQL> select * from test;

         A          C          B                                              
---------- ---------- ----------                                              
         1          2                                                        
         1          2          3                                              

SQL> insert into test(a,b,c) values (1,2,3);

1 row created.

SQL> select * from test;

         A          C          B                                              
---------- ---------- ----------                                              
         1          2                                                        
         1          2          3                                              
         1          3          2        

                                                             

Making Invisible column with NOT NULL constraint
---------------------------------------------------
SQL> alter table test modify c number not null;
Table altered.

SQL> alter table test modify (c invisible);
Table altered.

SQL> desc test
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 A                                                  NUMBER
 B                                                  NUMBER

SQL> select * from test;
         A          B                                                        
---------- ----------                                                        
         1                                                                    
         1          3                                                        
         1          2                                                        

SQL> insert into test values(1,2);
insert into test values(1,2)
*
ERROR at line 1:
ORA-01400: cannot insert NULL into ("SCOTT"."TEST"."C")

- Since column 'C' has not null constraint, default NULL value can not be inserted.

SQL> insert into test values(1,2,3);
insert into test values(1,2,3)
            *
ERROR at line 1:
ORA-00913: too many values

--- we can insert value to hidden column explicitly by specifying the column

SQL> insert into test(a,b,c) values (1,2,3);
1 row created.


SQL> select * from test;
         A          B                                                        
---------- ----------                                                        
         1                                                                    
         1          3                                                        
         1          2                                                        
         1          2                                                        

SQL> select a,b,c from test;
         A          B          C                                              
---------- ---------- ----------                                              
         1                     2                                              
         1          3          2                                              
         1          2          3                                              
         1          2          3                                              

-- To know hidden columns of a table query user_tab_cols

SQL> col table_name format a5
SQL> col column_name format a5
SQL> col hidden_column format a5
SQL> select table_name, column_name, hidden_column from user_tab_cols where table_name='TEST';

TABLE COLUM HIDDE                                                            
----- ----- -----                                                            
TEST  A     NO                                                                
TEST  B     NO                                                                
TEST  C     YES

Sunday, November 2, 2014

Oracle 12c new feature: Move datafile online

Prior to 12c, we have to put tablespace offline to move datafiles to new location and also we need to manually copy the file.

From 12c, it will be done automatically through online mode

Eg: Moving users tablespace datafile to different name at different location when it was online

SQL> select tablespace_name, file_name, status from dba_data_files where tablespace_name='USERS';

TABLESPACE_NAME
------------------------------
FILE_NAME
--------------------------------------------------------------------------------
STATUS
---------
USERS
/u01/app/oracle/oradata/NONCDB/datafile/users01.dbf
AVAILABLE


SQL> !ls -lrt /u01/app/oracle/oradata/NONCDB/datafile          
total 2736412
-rw-r-----. 1 oracle oinstall  91234304 Nov  3 06:45 o1_mf_temp_8x845rb3_.tmp
-rw-r-----. 1 oracle oinstall 450895872 Nov  3 06:50 o1_mf_undotbs1_8x843pdh_.dbf
-rw-r-----. 1 oracle oinstall 838868992 Nov  3 06:50 o1_mf_system_8x83zqv4_.dbf
-rw-r-----. 1 oracle oinstall 880812032 Nov  3 06:50 o1_mf_sysaux_8x83wgfw_.dbf
-rw-r-----. 1 oracle oinstall 627843072 Nov  3 06:53 users01.dbf

SQL> alter database move datafile '/u01/app/oracle/oradata/NONCDB/datafile/users01.dbf' to '/u01/app/oracle/oradata/NONCDB/users_renamed01.dbf';

Database altered.

SQL> !ls -lrt /u01/app/oracle/oradata/NONCDB/datafile
total 2123280
-rw-r-----. 1 oracle oinstall  91234304 Nov  3 06:45 o1_mf_temp_8x845rb3_.tmp
-rw-r-----. 1 oracle oinstall 450895872 Nov  3 06:50 o1_mf_undotbs1_8x843pdh_.dbf
-rw-r-----. 1 oracle oinstall 838868992 Nov  3 06:50 o1_mf_system_8x83zqv4_.dbf
-rw-r-----. 1 oracle oinstall 880812032 Nov  3 06:50 o1_mf_sysaux_8x83wgfw_.dbf

SQL> !ls -lrt /u01/app/oracle/oradata/NONCDB
total 633640
drwxr-x---. 2 oracle oinstall      4096 Jul  3  2013 controlfile
drwxr-x---. 2 oracle oinstall      4096 Jul  3  2013 onlinelog
-rw-r-----. 1 oracle oinstall  10493952 Nov  3 06:44 ilm1.dbf
-rw-r-----. 1 oracle oinstall  10493952 Nov  3 06:44 ilm2.dbf
-rw-r-----. 1 oracle oinstall 627843072 Nov  3 06:54 users_renamed01.dbf
drwxr-x---. 2 oracle oinstall      4096 Nov  3 06:54 datafile

SQL> select tablespace_name, file_name, status from dba_data_files where tablespace_name='USERS';

TABLESPACE_NAME
------------------------------
FILE_NAME
--------------------------------------------------------------------------------
STATUS
---------
USERS
/u01/app/oracle/oradata/NONCDB/users_renamed01.dbf
AVAILABLE

Saturday, October 25, 2014

Oracle 12.1.0.2 installation on RHEL/OL 7

Oracle 12.1.0.2 installation steps on Oracle Linux/RHEL 7

Prerequisites
Physical memory = 1024MB
Swap
Min - 1024 MB
Physical memory- 1024 to 2048 then swap 1.5*physical memory
Physical memory> 2048MB then swap=physical memory

Free Space - 6.4GB
temp area=1gB

User required:
INSTALL_USER
BACKUP_USER

Required groups:
INSTALL_GROUP
DBA_GROUP
OPER_GROUP  
BACKUPDBA_GROUP          
DGDBA_GROUP
KMDBA_GROUP
ASM_GROUP

Kernel parameters:
semmsl=250
semmns=32000
semopm=100
semmni=128
shmmax= 0.5*physical ram
shmmni=4096
shmall=physical ram*0.4
file-max=6815744
ip_local_port_range 9000 to 65535
rmem_default=262144
rmem_max=4194304
wmem_default=262144
wmem_max=1048576
aio-max-nr=1048576

OS Packages:

ocfs2 1.2.9
ocfs2-tools 1.2.7
binutils 2.23.52.0.1
compat-libcap1 1.10
compat-libstdc++-33 3.2.3    
libgcc 4.8.2  
libstdc++ 4.8.2
libstdc++-devel 4.8.2
sysstat 10.1.5
gcc 4.8.2
gcc-c++ 4.8.2
make 3.82
glibc 2.17
glibc-devel 2.17      
libaio 0.3.109
libaio-devel 0.3.109

Mount Parameters
nfs  options rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noac,vers=3,timeo=600/>
You can ignore if you don't want to configure OUI for MOS access













Sunday, September 7, 2014

12c Database Upgrade/Migrate using Full Transportable Export/Import
Traditional methods (Original export/import, Datapump export/import, Transportable tablespaces) can be used to upgrade/migrate database to 12c. In addition to that, 12c provides new feature called Full transportable export/import method to make migration/upgrade to easier, faster and more efficient.
Traditional export/import is slower method. For large database size systems it is not preferred method. Though it is a reliable migration technique, it should be used only for Oracle 9i and earlier databases.
Datapump export/import was introduced in Oracle 10g offers more features. It is designed to handle large volume of data.  But Datapump is overtaken by Transportable tablespace in terms of performance.
Transportable tablespaces achieves upgrade/migrate faster by copying database files to target system. Database file conversion will be required in case source and target has different Endian format. Since database file is directly copied it is faster than Datapump export/import. But it requires additional complicated set of steps to move user and system metadata to destination database. User objects associated with non user tablespaces (system and sysaux tablespaces) needs to be taken care manually.
Oracle 12c has introduced new feature “Full Transportable Export/Import”. It uses both Transportable tablespaces mechanism and Datapump features.  Transportable tablespace mechanism used to move database files to target system which results in migration in a faster even for large volume of data. Also it takes advantage of Datapump to take advantage of option to move metadata to target database and it is able to complete migration with single import command. At the end Full transportable export/import moves all of user data and metadata needed for database migration.
Prerequisites
1)      Create new target 12c database. It can non-CDB(Container database) or PDB (Pluggable database)
2)      Grant DATAPUMP_IMP_FULL_DATABASE role to user going to perform migration
3)      Set tablespaces read-only before performing upgrade/migrate
Steps
1) Create new 12c database:
The newly create database includes set of administrative tablespaces, Oracle supplied components and packages
2) Check Endianness of source and target platform
SELECT d.PLATFORM_NAME, ENDIAN_FORMAT FROM V$TRANSPORTABLE_PLATFORM tp, V$DATABASE d  WHERE tp.PLATFORM_ID = d.PLATFORM_ID;
3) Verify the tablespaces to be transported is self-contained
We must verify that there are no logical or physical dependencies between objects stored in tablespaces being transported and those that are not being transported
SQL> EXECUTE DBMS_TTS.TRANSPORT_SET_CHECK(' User Tablespace names separated by comma operator', TRUE);
The results can be checked at
SQL> SELECT * FROM TRANSPORT_SET_VIOLATIONS;
If query has returned rows then we need to address the issues before proceeding to full transportable operation
4) Create directory object
Create Directory object at source database to store dump
SQL> CREATE DIRECTORY dp_dir AS ’<Physical location>’;
5) Place user tablespaces which is to be transported in read only mode

SQL> ALTER TABLESPACE <tablespace_name> READ ONLY;  -- Downtime starts from here
6) Invoke full transportable export on the source database
expdp system/<password> full=y transportable=always version=12 \
directory=<database directory> dumpfile=<dump file name> \
metrics=y exclude=statistics \
logfile=<log file name>
 Version parameter is required only when source database is 11.2.0.3 or 11.2.0.4
7) Copy database datafiles to target site
Physically copy datafiles of tablespaces to target site
8) Create directory object at destination site
SQL> CREATE DIRECTORY dp_dir AS ’<Physical location>’;
9) Invoke full transport import at destination database
impdp system/<password>@<db name> directory=<directory name> \
dumpfile=<dump file name> logfile=<log file name> \
metrics=y \
transport_datafiles='<datafile names with full path>',

10) Convert tablespaces to read-write at source and target database - Downtime is completed
SQL> alter tablespace <tablespace-name> read write;