--- title: "Frameworks Monitoring Logs" canonical: "https://kb.myframeworks.com.au/space/FRAM/28396332/Frameworks%20Monitoring%20Logs" format: markdown --- # Overview While the DR is functioning normally, the AI Engine script, which is scheduled via root's cron at the defined **[SyncInterval],** sends an email of either success or failure to the defined **[MailAccount]**, and includes the main log file in the email. To receive these emails, you need to either configure a POP mailbox on your Frameworks Production server or configure your server to forward mail to your mail server, so that you receive instant alerts of the cycle's success or failure. In the event of either an email indicating a failure, or no email at all, it becomes necessary to investigate the possible cause of the problem by examining the various log files. These log files exist on both the Production and DR servers to ensure that normal database synchronisation is running successfully. Typical problems might be that the DR server was shutdown, or the communication link to it was lost. The log files are just text files; typical Unix commands such as cat and tail -f are used to check them. These various log files are outlined as follows: # Production Server Logs AI Engine Log The AI Engine script cp_next_ai writes to a new log file on each cycle, named **[AiEngineLog] ** where: **mmdd-HHMM** refers to the month, day, hour, and minute the time script starts. When the script first starts, it moves the previous log file to the log archive directory, so that a history is maintained in **[AiArchiveLogs]** This log file is fairly self-explanatory. It can be used to determine if the AI switch process has been started, in case the remote DR server needs to be taken down. ## Production DB Log The Production database log file **[LiveDbPath]**/stix.lg logs an entry each time the AI is switched to a new one, similar to the one below: | | | | | --- | --- | --- | | ^{[2008/08/07@09:00:03] P-28615} | ^{T-2432464 I Usr} | ^{130: (452)Login by root on batch.} | | ^{[2008/08/07@09:00:03] P-28615} | ^{T-2432464 I Usr} | ^{130: (7129) Usr 130 set name to Aimage} | | ^list. | | | | | | | | | --- | --- | --- | --- | | ^{[2008/08/07@09:00:03] P-28615} | ^{T-2432464 I Usr} | ^{130: (453)} | ^{Logout by Aimage list on} | | ^batch. | | | | | ^{[2008/08/07@09:00:03] P-28619} | ^{T-2430656 I Usr} | ^{130: (452)} | ^{Login by root on batch.} | | ^{[2008/08/07@09:00:03] P-28619} | ^{T-2430656 I Usr} | ^{130: (7129)} | ^{Usr 130 set name to Aimage} | | ^new. | | | | | ^{[2008/08/07@09:00:03] P-28619} | ^{T-2430656 I Usr} | ^{130: (3777)} | ^{Switched to ai extent} | | ^{/prostix/data/live/stix.a1.} | | | | | ^{[2008/08/07@09:00:03] P-28619} | ^{T-2430656 I Usr} | ^{130: (3778)} | ^{This is after-image file} | | ^{number 1504 since the last AIMAGE BEGIN} | | | | | ^{[2008/08/07@09:00:03] P-28619} | ^{T-2430656 I Usr} | ^{130: (453)} | ^{Logout by Aimage new on} | | ^batch. | | | | | ^{[2008/08/07@09:00:03] P-28622} | ^{T-1121392 I Usr} | ^{130: (452)} | ^{Login by root on batch.} | | ^{[2008/08/07@09:00:03] P-28622} | ^{T-1121392 I Usr} | ^{130: (7129)} | ^{Usr 130 set name to Aimage} | | ^full. | | | | | ^{[2008/08/07@09:00:03] P-28622} | ^{T-1121392 I Usr} | ^{130: (453)} | ^{Logout by Aimage full on} | | ^batch. | | | | # DR Server Logs The following log files exist on the remote DR server: ## AI Roll Forward Log Each time the AI Engine applies the AI file to the DR Live DB, it logs the Progress roll forward output to the log file **[DrDbPath]**/stix.roll The script that does this is the roll_forward_ai script, & it is the output from the actual Progress rfutil –C aimage roll forward command. Currently it's appended to, & is not purged. Log rotation may be added at a later date, but in the interim manual archiving on a periodic basis may be required. A sample follows: ``` Mon Aug 29 11:14:05 EST 2005 Working on AI file: /prostix/live/ai/bak/stix.a7.0829-1115 After-image dates for this after-image file: (1633) Last AIMAGE BEGIN Tue May 17 07:26:04 2005 (1640) Last AIMAGE NEW Mon Aug 29 10:15:03 2005 (1641) This is aimage file number 799 since the last AIMAGE BEGIN. (1642) This file was last opened for output on Mon Aug 29 10:15:03 2005. (1643) 1577 notes were processed. (1634) 0 in-flight transactions. (3785) 45 transactions were started 45 transactions were ended At the end of the .ai file, 0 of them were still active ``` ## DR Live DB Log Each time the roll forward runs, the Progress rfutil command also logs entries to the database file [DrDbPath]/stix.lg A sample follows: ``` Thu Aug 7 09:10:09 2008 [2008/08/07@09:10:09] P-17899T-1121392 I : (451)Roll forward session begin for root on batch. [2008/08/07@09:10:09] P-17899T-1121392 I : (5326) Begin Physical Redo Phase at 0 . [2008/08/07@09:10:10] P-17899T-1121392 I : (7161) Physical Redo Phase Completed at blk 9514 off 6423 upd 50565. [2008/08/07@09:10:10] P-17899T-1121392 I : (13547) At end of Physical redo, transaction table size is -(. [2008/08/07@09:10:10] P-17899T-1121392 I : (660)Beginning roll forward of after-image file /prostix/live/ai/bak/stix.a2.0807-0910. ``` | | | | | --- | --- | --- | | ^{[2008/08/07@09:10:10] P-17899} | ^{T-1121392 I : (1633)} | ^{After-image dates for this after-image} | | ^file: | | | | ^{[2008/08/07@09:10:10] P-17899} | ^{T-1121392 I : (1640)} | ^{Last AIMAGE BEGIN Tue Jul 1 03:33:25} | | ²⁰⁰⁸ | | | | ^{[2008/08/07@09:10:10] P-17899} | ^{T-1121392 I : (1641)} | ^{Last AIMAGE NEW Thu Aug 7 09:05:03} | | ²⁰⁰⁸ | | | | ^{[2008/08/07@09:10:10] P-17899} | ^{T-1121392 I : (1642)} | ^{This is aimage file number 1505 since} | | ^{the last AIMAGE BEGIN.} | | | ``` [2008/08/07@09:10:10] P-17899T-1121392 I : (1643)This file was last opened for output on Thu Aug 7 09:05:03 2008. [2008/08/07@09:10:10] P-17899T-1121392 I : (1634) 16827 notes were processed. [2008/08/07@09:10:10] P-17899T-1121392 I : (3785) 5 in-flight transactions. [2008/08/07@09:10:10] P-17899T-1121392 I : (1635) 604 transactions were started. [2008/08/07@09:10:10] P-17899T-1121392 I : (11138) 605 transactions were completed. [2008/08/07@09:10:10] P-17899T-1121392 I : (1636) At the end of the .ai file, 4 transactions were still active. [2008/08/07@09:10:10] P-17899T-1121392 I : (662)Roll forward completed. [2008/08/07@09:10:10] P-17899T-1121392 I : (334)rfutil -C roll forward session end. ``` ``` In addition, the drlive2demo.sh script, which refreshes the Demo/Test working copy database (see next log) also logs entries in this [DrDbPath]stix.lg log file, as the results from the Progress probkup -norecover commands. An example follows: Fri Jul 25 15:56:17 2008 [2008/07/25@15:56:17] P-6213 T-1121392 I: (451)Probkup session begin for root on batch. [2008/07/25@15:56:17] P-6213 T-1121392 I: (5581) Probackup session started with the norecover option in effect. [2008/07/25@15:56:17] P-6213 T-1121392 I: (6686) 20834713 active blocks out of 22000020 blocks in /prostix/data/live/stix will be dumped. [2008/07/25@15:56:17] P-6213 T-1121392 I: (6688) 24000 BI blocks will be dumped. [2008/07/25@15:56:17] P-6213 T-1121392 I: (9285) Backup requires an estimated 20.2 GBytes of media. [2008/07/25@15:56:17] P-6213 T-1121392 I: (9286) Restore would require an estimated 21026713 db blocks using 20.1 GBytes of media. [2008/07/25@15:56:17] P-6213 T-1121392 I: (12850) Backup blocks will be written to /tmp/pipe. [2008/07/25@15:56:17] P-6213 T-1121392 I: (1362) Full backup started. [2008/07/25@15:56:17] P-6213 T-1121392 I: (5459) Begin backup of Before Image file(s). [2008/07/25@15:56:20] P-6213 T-1121392 I: (5460) End backup of Before Image file(s). [2008/07/25@15:56:20] P-6213 T-1121392 I: (5461) Begin backup of Data file(s). [2008/07/25@16:03:02] P-6213 T-1121392 I: (5462) End backup of Data file(s). [2008/07/25@16:03:02] P-6213 T-1121392 I: (13625) Wrote a total of 623442 backup blocks using 20.2 GBytes of media. [2008/07/25@16:03:02] P-6213 T-1121392 I: (1364) Full backup successfully completed. [2008/07/25@16:03:03] P-6213 T-1121392 I: (334)Probkup session end. ``` ## DR Demo DB Refresh Log The drlive2demo.sh script which refreshes the Demo/Test working copy database on the DR server logs its output to the log file **[DrDbPath]**/stix.copy It's the output from the Progress probkup/prorest commands. ## DR Demo DB Log In addition, the drlive2demo.sh script also logs the prorest results in the DR Demo DB log file **[DrDbDemoPath]**/stix.lg, as follows: ^{Fri Jul 25 15:56:17 2008} | | | | | --- | --- | --- | | ^{[2008/07/25@15:56:17] P-6215} | ^{T-1121392 I} | ^{: (451)Prorest session begin for root on} | | ^batch. | | | | ^{[2008/07/25@15:56:19] P-6215} | ^{T-1121392 I} | ^{: (1368) Full restore started.} | | ^{[2008/07/25@15:56:19] P-6215} | ^{T-1121392 I} | ^{: (12854) Restoring database from} | | ^{/prostix/data/live/stix.db.} | | | | ^{[2008/07/25@16:03:02] P-6215} | ^{T-1121392 I} | ^{: (1369) Full restore completed.} | | ^{[2008/07/25@16:03:03] P-6215} | ^{T-1121392 I} | ^{: (334)Prorest session end.} |