Amazon Partner

Friday, 16 July 2010

Use and access SecureFile LOBS

SecureFiles is a Oracle database 11g New Feature to deliver scalability and high performance for unstructured or file data inside oracle. Its best of both world( file system and database). SecureFiles data can be updated/retrieved at same performance as traditional file system plus advantages of database(11g).

SecureFiles supported Features :

  • Deduplication - detecting identical securefile data, and storing only one copy
  • Compression - Saves storage, I/O, redo and encryption overhead.
  • Encryption - Uses TDE syntax, and current encryption algorithms.
  • Journaling - File system-like logging.



How to Configure/Use SecureFiles LOBS?

SecureFiles functionality is available in Oracle Database 11g by init.ora parameter db_securefile.

Default Value is "PERMITTED" - Allow SecureFile Creation.

Other Values are :
ALWAYS - Attempt to create SecureFile, if not possible, creates BasicFile LOB
FORCE  - Forces all LOBs created from that point forward to be SecureFiles
NEVER -  Disallows SecureFile LOB creation
IGNORE - Disallows SecureFile LOBs, and ignores errors if a BasicFile LOB is created with forced SecureFile optio

db_scurefile is a dynamic parameter and can be changed with alter system command.

for example, to force all LOBs creation in Securefiles.

SQL > alter system set db_securefile='FORCE' scope=both sid='*';



alter system set db_securefile='PERMITTED' SCOPE=BOTH;

create table SECURE_FILES_M_PERMITTED
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS01_MANUAL);


create table SECURE_FILES_M_PERMITTED


*
ERROR at line 1:
ORA-43853: SECUREFILE lobs cannot be used in non-ASSM tablespace "TBS01_MANUAL"



create table SECURE_FILES_A_PERMITTED
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS02_AUTO);

Table created.



alter system set db_securefile='FORCE' SCOPE=BOTH;

create table SECURE_FILES_M_FORCE
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS01_MANUAL);


create table SECURE_FILES_M_FORCE
*
ERROR at line 1:
ORA-43853: SECUREFILE lobs cannot be used in non-ASSM tablespace "TBS01_MANUAL"


create table SECURE_FILES_A_FORCE
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS02_AUTO);

Table created.



alter system set db_securefile='ALWAYS' SCOPE=BOTH;

create table SECURE_FILES_M_ALWAYS
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS01_MANUAL);


create table SECURE_FILES_M_ALWAYS
*
ERROR at line 1:
ORA-43853: SECUREFILE lobs cannot be used in non-ASSM tablespace "TBS01_MANUAL"


create table SECURE_FILES_A_ALWAYS
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS02_AUTO);

Table created.


alter system set db_securefile='NEVER' SCOPE=BOTH;

create table SECURE_FILES_M_NEVER
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS01_MANUAL);

Table created.


create table SECURE_FILES_A_NEVER
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS02_AUTO);

Table created.


alter system set db_securefile='IGNORE' SCOPE=BOTH;

create table SECURE_FILES_M_IGNORE
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS01_MANUAL);

Table created.


create table SECURE_FILES_A_IGNORE
(FILE_NAME VARCHAR2(30),
FILE_ID NUMBER,
FILE_DATA BLOB)
LOB(FILE_DATA) STORE AS SECUREFILE (TABLESPACE TBS02_AUTO);


Table created.

Analysis report based on above test. 


Above table can be used to understand the combination of parameters value/Storage management type and LOB type creation impact.

Conclusion :
So if you want to create SecureFile Storage management must be ASSM and value of db_scurefile parameter must be "ALWAYS|FORCE|PERMITTED".

Tuesday, 25 May 2010

Disable / Enable Database Vault 11g


To Disable or Enable Database Vault you need to stop the oracle home and related process before hand.


Disable Oracle Database Vault as follows:





  • UNIX: 
    $ cd $ORACLE_HOME/rdbms/lib
    $ make -f ins_rdbms.mk dv_off ioracle
    
    
    


  • Windows: In the ORACLE_BASE\ORACLE_HOME\bin directory, rename the oradvll.dll file to another name, such as oradvll.dll.dbl.




Enable Oracle Database Vault as follows:




  • UNIX: Run the following commands. The make command enables both Oracle Database Vault (dv_on) and Oracle Label Security (lbac_on). You must have Oracle Label Security enabled before you can use Database Vault.

    $ cd $ORACLE_HOME/rdbms/lib
    $ make -f ins_rdbms.mk dv_on lbac_on ioracle
    
  • Windows: In the ORACLE_BASE\ORACLE_HOME\bin directory, rename the backed up copy of the oradvll.dll file (for example, oradv11.dll.dbl) tooradvll.dll. Ensure that the name of the Oracle Label Security executable is oralbacll.dll (and not oralbacll.dll.dbl or some other backup name). You must have Oracle Label Security enabled before you can use Oracle Database Vault.

Grid Control 11.1.0.1 Administration Start/Stop

Grid Control 11.1. Released Last month, Struggled to find the right instruction to start and stop the environment, Following Steps can be used to start/Stop grid control 11.1.0.1


Start Sequence:
1. Start Listeners <= Start all the listeners
2. Start database <= start the database
3. Start OMS <= This will start WebTier and OMS
  
4. Start Agent <= Start agent

Stop grid control : Stop sequence will be opposite to start.
1. Stop agent
2. Stop OMS
3. Stop database
4. Stop Listeners


:::::START ::::
1. Start Listeners
$ . oraenv
ORACLE_SID = [oracle] ? gcdb01
The Oracle base for ORACLE_HOME=/u01/app/product/11.2.0/dbhome_1 is /u01/app/oracle


lsnrctl start

2. Start Database
$ . oraenv
ORACLE_SID = [oracle] ? gcdb01
The Oracle base for ORACLE_HOME=/u01/app/product/11.2.0/dbhome_1 is /u01/app/oracle
$ sqlplus "/ as sysdba"
SQL > startu
p

3. Start OMS 

$ . oraenv
ORACLE_SID = [gcdb01] ? oms11g
The Oracle base for ORACLE_HOME=/u01/app/product/Middleware/oms11g is /u01/app/oracle

$ $ORACLE_HOME/bin/emctl start oms
 






$ORACLE_HOME/bin/emctl status oms -details
Oracle Enterprise Manager 11g Release 1 Grid Control
Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :>
Console Server Host : 254739-andb3.kk.local
HTTP Console Port   : 7788
HTTPS Console Port  : 7799
HTTP Upload Port    : 4889
HTTPS Upload Port   : 4900
OMS is not configured with SLB or virtual hostname
Agent Upload is locked.
OMS Console is locked.
Active CA ID: 1


4. Start agent
$ . oraenv
ORACLE_SID = [oms11g] ? agent11g
The Oracle base for ORACLE_HOME=/u01/app/product/Middleware/agent11g is /u01/app/oracle




$ $ORACLE_HOME/bin/emctl start agent

-------------------------------------------------------------------

:::::STOP SEQUENCE ::::

1. Stop agent
$ . oraenv
ORACLE_SID = [oms11g] ? agent11g
The Oracle base for ORACLE_HOME=/u01/app/product/Middleware/agent11g is /u01/app/oracle

$
$ORACLE_HOME/bin/emctl stop agent


Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
Stopping agent ... stopped.



2. Stop OMS Components

$ . oraenv
ORACLE_SID = [gcdb01] ? oms11g
The Oracle base for ORACLE_HOME=/u01/app/product/Middleware/oms11g is /u01/app/oracle

$ $ORACLE_HOME/bin/emctl stop oms
-all




Oracle Enterprise Manager 11g Release 1 Grid Control


Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
Stopping WebTier...
WebTier Successfully Stopped
Stopping Oracle Management Server...
Error Occurred: Error during stop oms. Please check error and log files


--------------------------------------------------------------------------------------------------------
/u01/app/product/gc_inst/em/EMGC_OMS1/sysman/log/emctl.log
----------------------------------------------------------------------------------------------------------



2010-05-25 08:58:45,110 [Thread-1] INFO  wls.OMSController run.1094 - Failed to stop admin server
2010-05-25 08:58:45,114 [Thread-1] INFO  wls.OMSController run.1094 -
2010-05-25 08:58:45,115 [Thread-1] INFO  wls.OMSController run.1094 - _END_
2010-05-25 08:58:45,115 [Thread-2] INFO  wls.OMSController run.1094 - _END_
2010-05-25 08:58:45,149 [Main Thread] INFO  wls.OMSController processCmd.988 - wlst process is finished
2010-05-25 08:58:45,150 [Main Thread] INFO  wls.OMSController processCmd.990 - wlst process exited with code 125
2010-05-25 08:58:45,151 [Main Thread] INFO  wls.OMSController processCmd.992 - wlst outstream joined
2010-05-25 08:58:45,151 [Main Thread] INFO  wls.OMSController processCmd.994 - wlst errstream joined
2010-05-25 08:58:45,152 [Main Thread] ERROR wls.OMSController main.258 - OMSController failed for stop oms
2010-05-25 08:58:45,154 [Main Thread] ERROR wls.OMSController main.259 - OMSController Error: Error during stop oms. Please check error and log files
java.lang.Exception: Error during stop oms. Please check error and log files
        at oracle.sysman.emctl.wls.OMSController.stopOMS(OMSController.java:812)
        at oracle.sysman.emctl.wls.OMSController.main(OMSController.java:240)
------------------------------------------------------------------------------------


3. Stop Database
$ . oraenv
ORACLE_SID = [oracle] ? gcdb01
The Oracle base for ORACLE_HOME=/u01/app/product/11.2.0/dbhome_1 is /u01/app/oracle
$ sqlplus "/ as sysdba"



SQL*Plus: Release 11.2.0.1.0 Production on Tue May 25 09:02:30 2010
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options





4. Stop Listeners
$ . oraenv
ORACLE_SID = [oracle] ? gcdb01
The Oracle base for ORACLE_HOME=/u01/app/product/11.2.0/dbhome_1 is /u01/app/oracle
 $ORACLE_HOME/bin/lsnrctl stop LISTENER_APP


LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 25-MAY-2010 09:04:58
Copyright (c) 1991, 2009, Oracle.  All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=254739-andb3.kkj.local)(PORT=61902)))
The command completed successfully




Friday, 11 December 2009

Oracle 11g or 11gR2 Tablespace Encryption

Oracle 11g (11.2.0.1) Configure Tablespace Encryption using Safenet Hardware Security Module (LUNA SA 4.4.0 HSM)

Note: Step 1 to 4 need repeating on all the nodes in the cluster

Step 5 and 6 don't need repeat for RAC

Example : DATABASE SERVER Hostname is dbserver01
: HSM Device name is hsm01


1. Install HSM Client software on database Server dbserver01:


2. Register Oracle Database server (client) with HSM.

login as root on dbserver01:

su - root
password: *******

[root@dbserver01 bin]# cd /usr/lunasa/bin

-- Fetch server parameter from hsm01 to database server.

[root@dbserver01 bin]# /ctp admin@hsm01.domain.local:server.pem .

--- Verify Cerfiticate
[root@dbserver01 bin]# openssl x509 -in server.pem -noout -text

--- Register Hsm01 as authorised HSM Server.

[root@dbserver01 bin]# ./vtl addServer -n hsm01 -c server.pem

New server hsm01 successfully added to server list.

-- Verify Server added

[root@dbserver01 bin]# ./vtl listServers
Server: hsm01


-- Create HSM Client certificate from the Database Server

[root@dbserver01 bin]# ./vtl createCert -n dbserver01
Private Key created and written to: /usr/lunasa/cert/client/dbserver01Key.pem
Certificate created and written to: /usr/lunasa/cert/client/dbserver01.pem


--- upload client certificate to HSM

[root@dbserver01 bin]# ./ctp /usr/lunasa/cert/client/dbserver01.pem admin@hsm01.domain.local:
admin@hsm01.domain.local's password:
dbserver01.pem 100% 1172 1.1KB/s 00:00


---- Login to HSM Device via SSH or Local console

ssh admin@hsm01.domain.local
password: ***********

----- Register dbserver01 to HSM as client

[hsm01] lunash:>client reg -c dbserver01 -h dbserver01

'client register' successful.

Command Result : 0 (Success)


--- Assing HSM partition to dbserver01(client)

[hsm01] lunash:>client a -p Oracle -c dbserver01

'client assignPartition' successful.

Command Result : 0 (Success)


3. Verify that Oracle Database server can talk to HSM:


[root@gridcluster03 bin]# ./vtl verify

The following Luna SA Slots/Partitions were found:

Slot Serial # Label
==== ======== =====
1 951962001 Oracle

---- You have Successfully Added Database server to HSM as client and Now Oracle Server is ready to use HSM as wallet to Store Tablespace and Column Master keys.



4. Configure Oracle to use HSM.

--- copy the HSM Libarary to Oracle required location:

[root@dbserver01 bin]# mkdir -p /opt/oracle/extapi/64/hsm/safenet/4.4.0/

[root@dbserver01 bin]#cp /usr/lunasa/lib/libshim.so /opt/oracle/extapi/64/hsm/safenet/4.4.0/libshim.so

[root@dbserver01 bin]#chown -R oracle:dba /opt/oracle


--- Login as oracle and update sqlnet.ora

[root@dbserver01 bin]# su - oracle

[oracle@dbserver01 ]$ vi $ORACLE_HOME/network/admin/sqlnet.ora


Add the following line
------------------------------
ENCRYPTION_WALLET_LOCATION = (SOURCE = (METHOD = HSM))


5. Enable Tablespace Encryption in database by createing the Master key in HSM wallet.


[oracle@dbserver01 ]$ sqlpus "/ as sysdba"


----- Create wallet in HSM Use this command once only to create the master key. Every time you bounce database you need to open the wallet.

alter system set encryption key identified by "hsmpartition_password";



---- Create tablespace:

CREATE TABLESPACE TDE_TEST01 DATAFILE '+DATA' SIZE 10m ENCRYPTION using 'AES256' DEFAULT STORAGE (ENCRYPT);




6. Every Time you open start database open the wallet by following command.

alter system set encryption wallet open identified by "hsmpartition_password";


Note: Step 1 to 4 need repeating on all the nodes in the cluster

Flashback Database

SQL> alter system set db_recovery_file_dest='+BACKUPDG' scope=both sid='*';

SQL> alter system set db_recovery_file_dest_size=5000M scope=both sid='*';

SQL> alter system set db_flashback_retention_target= 1440 scope=both sid='*';

--- Verify the parameter setup .

COLUMN NAME FORMAT A40
COLUMN VALUE FORMAT A50

SELECT INST_ID,NAME,VALUE FROM gv$PARAMETER WHERE NAME IN ('db_recovery_file_dest','db_recovery_file_dest_size','db_flashback_retention_target')
/

INST_ID NAME VALUE
---------- ---------------------------------------- --------------------------------------------------
1 db_recovery_file_dest +BACKUPDG
1 db_recovery_file_dest_size 5218762752
1 db_flashback_retention_target 1440
2 db_recovery_file_dest +BACKUPDG
2 db_recovery_file_dest_size 5218762752
2 db_flashback_retention_target 1440

6 rows selected.

-- Shutdown database and startup mount;

SQL> shutdown immediate;
SQL> startup mount;
Database mounted.

--- Ensure database is in archive log mode.

SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 71
Next log sequence to archive 0
Current log sequence 72

---- Turn the Flashback on;

SQL> alter database flashback on;

-- Verify Flashback in Turned on .

SQL> SELECT FLASHBACK_ON FROM V$DATABASE;

FLASHBACK_ON
------------------
YES

SQL> alter database open;

SQL> CREATE RESTORE POINT BEFORE_PATCH;

Restore point created.

---- Flashback Related Views :

SELECT OLDEST_FLASHBACK_SCN,OLDEST_FLASHBACK_TIME ,FLASHBACK_SIZE FROM v$flashback_database_log;

OLDEST_FLASHBACK_SCN OLDEST_FL FLASHBACK_SIZE
-------------------- --------- --------------
859379 10-DEC-09 159416320


SQL> SELECT SCN,NAME ,TIME FROM V$RESTORE_POINT;

SCN NAME TIME
---------- ---------------------------------------- ---------------------------------------------------------------------------
976812 BEFORE_PATCH 11-DEC-09 03.54.59.000000000 PM


COLUMN NAME FORMAT A50
1* SELECT NAME,LOG#,BYTES,FIRST_TIME FROM V$FLASHBACK_DATABASE_LOGFILE
SQL> /

NAME LOG# BYTES FIRST_TIM
-------------------------------------------------- ---------- ---------- ---------
+BACKUPDG/armynet/flashback/log_1.367.705266421 1 15941632 10-DEC-09
+BACKUPDG/armynet/flashback/log_2.365.705266477 2 15941632 10-DEC-09
+BACKUPDG/armynet/flashback/log_3.361.705276043 3 15941632 10-DEC-09
+BACKUPDG/armynet/flashback/log_4.360.705286825 4 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_5.358.705286845 5 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_6.357.705301241 6 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_7.356.705304845 7 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_8.354.705315645 8 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_9.349.705322797 9 15941632 11-DEC-09
+BACKUPDG/armynet/flashback/log_10.348.705330031 10 15941632 11-DEC-09

10 rows selected.



=======================

---- Flasback to scn:

FLASHBACK [STANDBY] DATABASE TO [BEFORE] SCN ;


---- Flasback to timestamp

FLASHBACK [STANDBY] DATABASE TO [BEFORE] TIMESTAMP ;


---- Flasback to restore_point
FLASHBACK [STANDBY] DATABASE TO [BEFORE] RESTORE POINT ;



Example of Flasback using Restore point (Standby database open read/write and convert back to standby)



CREATE RESTORE POINT before_open;

ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY;

ALTER DATABASE OPEN;


--- DATABASE OPEN TO READ/WRITE OPERATION.


----TO CONVERT BACK TO STANDBY;


SHUTDOWN IMMEDIATE;

STARTUP MOUNT;
ALTER DATABASE FALSHBACK TO RESTORE POINT BEFORE_OPEN;

ALTER DATABASE CONVERT TO PHYSICAL STANDBY;

RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

Tuesday, 8 December 2009

ORA-28362: master key not found

ORA-28362: master key not found
Cause:
The required master key required could not be located. This may be casued by the use of an invalid or incorrect wallet.
Action: Check wallet location parameters to see if they specify the correct wallet. Also, verify that an SSO wallet is not being used when an encrypted wallet is intended.

Error Indicate that It doesn't found the current Master key. Effectively Command will create the New master key as the specified location(Either FILE or HSM. Any objects(TAblespace) created with OLD Master key will be unusable.

ORA-28386: "cannot perform a PKI certificate-based Master Key Rekey"
// *Cause: A PKI certificate-based Master Key Rekey was attempted when
// Tablespace Encryption or HSM was in use.
// *Action: Perform a symmetric Master Key Rekey or disable Tablespace
// Encryption for all tablespaces.

11gR2 (11.2.0.1) database don't support REKEY if MASTERKEY is stored in HSM(Hardware Secure module".