Saturday, April 12, 2014

Rdbms-- Standby ORA-00600: internal error code, arguments: [krastrso_rectype]

When implementing Dataguard for a 11gR2 Oracle Database, you can encounter and ORA-00600[krastrso_rectype] on Standby Database.
This error prevents archived logs to be transferred to the Standby Site.
Actually , the RFS process triggers this error..
What 's happening in here is, the archived log file is transffered from the network, and written to the location pointed by Standby Database's log_archive_dest_1 parameter.
Following picture describes the flow; (the red circle is the problematic point)


RFS writes the blocks, but encounters and error while closing the file.. ORA-00600 [krastrso_rectype]
Unfortuneatly, there aint no information in Google or Oracle Support(Lookup tool or knowledge base) about this error.

Even when you enable maximum trace for RFS , LGWR or ARCH processes, you can not find any extra information about the error. You can only say that there is an error produced on the standby site, while closing the transferred archived log..

Actually, I have faced with this problem in one of my clients. I was called to analyze the problem, as this environment was not created by me or by my company.
In other words, I was involved the project subsequently..,
When I was there, I analyzed the situation, gathered process traces, checked Linux OS , searched google and Support , but couldnt find any info.
I knew that there is some problem closing the transffered archived log file on standby site, but couldnt find a reason directly, because I supposed the installation was proper..
After digging for several hours, I tried to change the log_archive_dest_1 parameter in standby database. It was pointing the db_recovery_file_dest, so I changed it to be pointed to a normal directory..(/u1/erman)
Suddenly, standby database stopped producing the ORA-600 and started working properly..
That was  a clue for me, and that was also a workaround, too..
Then I decided the find the problem lying behind... Because it was abnormal.. use_db_recovery_dest could be used in standby database normally without any issue.
I checked the parameter file on standby database and saw that db_file_name_convert and log_file_name_convert parameters were set..

Lets identify those parameters; 

DB_FILE_NAME_CONVERT converts the filename of a new datafile on the primary database to a filename on the standby database. If you add a datafile to the primary database, you must add a corresponding file to the standby database. When the standby database is updated, this parameter converts the datafile name on the primary database to the datafile name on the standby database. 

LOG_FILE_NAME_CONVERT converts the filename of a new log file on the primary database to the filename of a log file on the standby database. If you add a log file to the primary database, you must add a corresponding file to the standby database.

Actually; this is the important information : 

The initialization parameters DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT cannot be used to rename OMF (non-ASM) file names for the auxiliary instance because this method generates invalid OMF file names. If you must control the generation of new OMF file names that do not use ASM storage, you must rename them using one of the following alternate techniques. The various naming options are listed in order from most recommended to least recommended.

Also , lets look at the limitations of db_file_name_convert and log_file_name_convert.
Usage and Limitation of db_file_name_convert and log_file_name_convert (Doc ID 1367014.1)

In this document , Oracle says;

It cannot be used for ;

Logical Standby Databases - only log_file_name_convert is possible for Logical Standby Databases
RMAN Restore - RMAN Duplicate/TSPITR only
If Oracle Managed Files (OMF) is used
To rename existing Datafiles, eg. when you move a Database. The Conversion only takes Place on File Creation on the Database where it is set.


And in OMF documents; we see that db_recovery_file_dest parameter is enabling the OMF. 

Enabling the Creation and Use of Oracle Managed Files

The following table lists the initialization parameters that enable the use of Oracle Managed Files.
Initialization ParameterDescription
DB_CREATE_FILE_DESTDefines the location of the default file system directory or Oracle ASM disk group where the database creates data files or temp files when no file specification is given in the create operation. Also used as the default location for redo log and control files ifDB_CREATE_ONLINE_LOG_DEST_n are not specified.
DB_CREATE_ONLINE_LOG_DEST_nDefines the location of the default file system directory or Oracle ASM disk group for redo log files and control file creation when no file specification is given in the create operation. By changing n, you can use this initialization parameter multiple times, where n specifies a multiplexed copy of the redo log or control file. You can specify up to five multiplexed copies.
DB_RECOVERY_FILE_DESTDefines the location of the Fast Recovery Area, which is the default file system directory or Oracle ASM disk group where the database creates RMAN backups when no format option is used, archived logs when no other local destination is configured, and flashback logs. Also used as the default location for redo log and control files or multiplexed copies of redo log and control files if DB_CREATE_ONLINE_LOG_DEST_n are not specified.


So, in conclusion, 

the issue was fixed by deleting log_file_name_convert and db_file_name_convert parameters in standby's parameter file, as those parameters make Oracle to behave unexpectedly in closing transffered acrhivelog files on the remote file system..

This seems to be an expected behaviour but ORA-00600 scared us :)
After removing those parameters, we could use db_recovery_file_dest as our log_archive_dest in standby database..

No comments :

Post a Comment