Oracle Cloud Infrastructure (OCI) Logging Analytics provides a comprehensive solution for managing and analyzing logs, facilitating easier troubleshooting, auditing, and monitoring of applications and infrastructure. In the cloud environment, it's crucial to use it effectively to optimize both performance and cost.
As a log analysis and log correlation cloud service, we always face requirements to ingest log data into Logging Analytics for log processing and indexing. There are many approaches to ingest your logs into Oracle Logging Analytics using the Oracle Management Agent, uploading logs on-demand, collecting them from the OCI Object Store, or using Service Connector. Based on the log sources and destination, we need to select the proper agent for various log ingestion use cases which ensures efficient collection is configured to filter, parse, and forward logs in a structured manner.
In my blog Demystifying Logging and Monitoring Agent Types in OCI Observability and Management, I have discussed different agent types and their use cases in OCI. It's very important to understand their feature parity and their pros and cons when choosing the agent for the job. As a common strategy, we make our agent decisions based on the destination service we are going to ingest logs into.
Ingesting logs via Unified Monitoring Agent(FluentD)
Ingesting logs via Management Agent
Once you have successfully picked the right agent for the right job and configured the parser and log source for the agent to ingest and process logs in OCI, you might face a challenge when the cloud resources generate a large amount of log data which may exceed the OCI Management Agent upload limit. In those cases, additional Management Agent performance tuning is required to make sure the Management Agent can push through as much log data as possible in real-time.
Fun facts about Management Agent configuration and Performance Tuning
In order to improve the Logging Analytics ingestion performance from the OCI Logging Service and Service Connector Hub, we can reduce Log Volume for logs
Another log collection theme in OCI O&M is to forward your logs to the central syslog server
Optimizing the Management Agent for peak performance in log ingestion is crucial for your infrastructure and operational teams. This enhancement is key to ensuring timely log analysis and detection. By adopting these best practices, you're not just adjusting a system; you're unlocking the full potential of your log data. This enables more informed decision-making, streamlines operational workflows, and boosts security measures. Embrace the power of Management Tuning in your log data analysis to elevate your organization's efficiency and performance, paving the way for operational superiority.
Please check out our Oracle Cloud Customer Connect Observability and Management Community. You can pose questions, connect with experts, and share your successes, thoughts and ideas about Oracle Cloud Observability and Management solutions (including OCI Application Performance Monitoring, Stack Monitoring, Logging Analytics, Database Management and Operations Insights).
Royce Fu is the Principal Database Solution Architect of the North America Cloud Technology and Engineering Team. Royce's area of specialty is core Database Technology and OCI O&M especially in Database Platform Engineering, Architecture, and Integration. He started his career as Java software engineer and spent over a decade in database engineering and architecture.