從零散文本到關(guān)聯(lián)可觀測性：Serilog與OpenTelemetry重塑.NET應(yīng)用調(diào)試體驗
作者：架構(gòu)師老盧
Serilog + OpenTelemetry不僅僅是更好的日志記錄——它是一種可觀測性，改變了您理解和調(diào)試.NET應(yīng)用程序的方式。
適用于現(xiàn)代.NET應(yīng)用程序的Serilog和OpenTelemetry架構(gòu)
當(dāng)您的.NET應(yīng)用程序在生產(chǎn)環(huán)境凌晨3點拋出一個難以理解的錯誤時，您最不愿意做的事情就是翻閱成千上萬的非結(jié)構(gòu)化日志文件，試圖拼湊出問題所在。傳統(tǒng)的日志記錄感覺就像大海撈針——不同的是，這個"草堆"可能正在著火，而且那根"針"甚至可能不存在。
Serilog和OpenTelemetry登場：這對強(qiáng)力組合將日志記錄從一種必要的麻煩轉(zhuǎn)變?yōu)槔斫夥植际较到y(tǒng)的秘密武器。
傳統(tǒng)日志記錄與結(jié)構(gòu)化日志記錄對比
傳統(tǒng)日志記錄的問題
想象一下：您的微服務(wù)架構(gòu)橫跨15個不同的服務(wù)，每個服務(wù)都像這樣輸出日志：
2025-09-10 14:32:17 INFO: Processing request for user John
2025-09-10 14:32:18 ERROR: Database timeout occurred
2025-09-10 14:32:19 INFO: Retrying operation現(xiàn)在回答這些問題：
? 哪個用戶觸發(fā)了錯誤？
? 原始請求是什么？
? 哪個服務(wù)實際失敗了？
? 整個請求花了多長時間？
使用傳統(tǒng)日志記錄，您就像在用不完整的證據(jù)進(jìn)行偵探工作。
為什么Serilog + OpenTelemetry是游戲規(guī)則改變者
使用Serilog進(jìn)行結(jié)構(gòu)化日志記錄
Serilog不是轉(zhuǎn)儲文本，而是創(chuàng)建機(jī)器可以理解的結(jié)構(gòu)化數(shù)據(jù)：
// 傳統(tǒng)方式（不佳）
_logger.LogInformation($"User {userId} ordered {itemCount} items for ${totalAmount}");
// Serilog結(jié)構(gòu)化方式（佳）
_logger.LogInformation("User {UserId} completed order {OrderId} with {ItemCount} items for {TotalAmount:C}",
userId, orderId, itemCount, totalAmount);這會生成如下所示的JSON：
{
"timestamp":"2025-09-10T14:32:17.123Z",
"level":"Information",
"messageTemplate":"User {UserId} completed order {OrderId} with {ItemCount} items for {TotalAmount:C}",
"message":"User john.doe completed order ORD-12345 with 3 items for $299.99",
"properties":{
"UserId":"john.doe",
"OrderId":"ORD-12345",
"ItemCount":3,
"TotalAmount":299.99
}
}現(xiàn)在您可以查詢："顯示所有超過200美元的訂單"或"查找用戶john.doe的所有錯誤"。
OpenTelemetry：缺失的一環(huán)
OpenTelemetry添加了關(guān)聯(lián)層，連接整個分布式系統(tǒng)中的日志。每條日志都會自動豐富以下信息：
? TraceId：跨所有服務(wù)跟蹤單個用戶請求
? SpanId：標(biāo)識該請求中的特定操作
? 服務(wù)上下文：哪個服務(wù)、版本和環(huán)境
設(shè)置這對強(qiáng)力組合
步驟1：安裝所需的NuGet包
dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Sinks.OpenTelemetry
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol步驟2：配置您的Program.cs
以下是提供具有完整可觀測性的結(jié)構(gòu)化日志記錄的完整設(shè)置：
using Serilog;
using OpenTelemetry.Logs;
using OpenTelemetry.Metrics;
using OpenTelemetry.Trace;
// 首先配置Serilog
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Information()
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
.Enrich.FromLogContext()
.Enrich.WithProperty("Application", "YourAppName")
.Enrich.WithProperty("Environment", Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT"))
.WriteTo.Console(new JsonFormatter()) // 結(jié)構(gòu)化控制臺輸出
.WriteTo.OpenTelemetry(options =>
{
options.Endpoint = "http://localhost:4317"; // OTLP端點
options.Protocol = OtlpProtocol.Grpc;
options.ResourceAttributes = new Dictionary<string, object>
{
["service.name"] = "your-service-name",
["service.version"] = "1.0.0"
};
})
.CreateLogger();
var builder = WebApplication.CreateBuilder(args);
// 使用Serilog進(jìn)行日志記錄
builder.Host.UseSerilog();
// 配置OpenTelemetry
builder.Services.AddOpenTelemetry()
.WithTracing(tracing => tracing
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddEntityFrameworkCoreInstrumentation() // 如果使用EF Core
.AddOtlpExporter(options =>
{
options.Endpoint = new Uri("http://localhost:4317");
}))
.WithMetrics(metrics => metrics
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddOtlpExporter(options =>
{
options.Endpoint = new Uri("http://localhost:4317");
}));
var app = builder.Build();
// 添加請求日志記錄中間件
app.UseSerilogRequestLogging(options =>
{
options.MessageTemplate = "HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms";
options.EnrichDiagnosticContext = (diagnosticContext, httpContext) =>
{
diagnosticContext.Set("RequestHost", httpContext.Request.Host.Value);
diagnosticContext.Set("RequestScheme", httpContext.Request.Scheme);
diagnosticContext.Set("UserAgent", httpContext.Request.Headers["User-Agent"].FirstOrDefault());
// 添加自定義業(yè)務(wù)上下文
if (httpContext.User.Identity.IsAuthenticated)
{
diagnosticContext.Set("UserId", httpContext.User.FindFirst("sub")?.Value);
}
};
});
app.Run();步驟3：設(shè)置OpenTelemetry Collector
創(chuàng)建docker-compose.yml以運行本地可觀測性堆棧：
version: '3.8'
services:
# OpenTelemetry Collector
otel-collector:
image:otel/opentelemetry-collector-contrib:latest
container_name:otel-collector
command: ["--config=/etc/otel-collector-config.yaml"]
volumes:
-./otel-collector-config.yaml:/etc/otel-collector-config.yaml
ports:
-"4317:4317" # OTLP gRPC接收器
-"4318:4318" # OTLP HTTP接收器
-"8889:8889" # Prometheus指標(biāo)
depends_on:
-jaeger
-prometheus
# Jaeger用于追蹤
jaeger:
image:jaegertracing/all-in-one:latest
container_name:jaeger
ports:
-"16686:16686"
-"14250:14250"
environment:
-COLLECTOR_OTLP_ENABLED=true
# Prometheus用于指標(biāo)
prometheus:
image:prom/prometheus:latest
container_name:prometheus
ports:
-"9090:9090"
volumes:
-./prometheus.yml:/etc/prometheus/prometheus.yml
# Grafana用于可視化
grafana:
image:grafana/grafana:latest
container_name:grafana
ports:
-"3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin創(chuàng)建otel-collector-config.yaml：
receivers:
otlp:
protocols:
grpc:
endpoint:0.0.0.0:4317
http:
endpoint:0.0.0.0:4318
processors:
batch:
timeout:1s
send_batch_size:1024
resource:
attributes:
-key:environment
value:development
action:upsert
exporters:
# 將追蹤導(dǎo)出到Jaeger
jaeger:
endpoint:jaeger:14250
tls:
insecure:true
# 將指標(biāo)導(dǎo)出到Prometheus
prometheus:
endpoint:"0.0.0.0:8889"
# 將日志導(dǎo)出到控制臺（您可以在此處添加Loki）
logging:
loglevel:debug
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch, resource]
exporters: [jaeger]
metrics:
receivers: [otlp]
processors: [batch, resource]
exporters: [prometheus]
logs:
receivers: [otlp]
processors: [batch, resource]
exporters: [logging]啟動堆棧：
docker-compose up -d高級日志記錄模式
1. 使用作用域的上下文日志記錄
添加強(qiáng)制應(yīng)用于作用域內(nèi)所有日志的業(yè)務(wù)上下文：
public classOrderService
{
privatereadonly ILogger<OrderService> _logger;
public async Task ProcessOrderAsync(int orderId, string userId)
{
// 創(chuàng)建帶有上下文的日志記錄作用域
usingvar scope = _logger.BeginScope(new Dictionary<string, object>
{
["OrderId"] = orderId,
["UserId"] = userId,
["Operation"] = "ProcessOrder"
});
_logger.LogInformation("Starting order processing");
try
{
await ValidateOrderAsync(orderId);
await ChargePaymentAsync(orderId);
await FulfillOrderAsync(orderId);
_logger.LogInformation("Order processing completed successfully");
}
catch (Exception ex)
{
_logger.LogError(ex, "Order processing failed");
throw;
}
}
}此作用域內(nèi)的每條日志都會自動包含OrderId、UserId和Operation。
2. 用于業(yè)務(wù)上下文的自定義擴(kuò)展器
創(chuàng)建添加一致業(yè)務(wù)上下文的擴(kuò)展器：
public classTenantEnricher : ILogEventEnricher
{
privatereadonly IHttpContextAccessor _contextAccessor;
public TenantEnricher(IHttpContextAccessor contextAccessor)
{
_contextAccessor = contextAccessor;
}
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
var context = _contextAccessor.HttpContext;
if (context?.User?.Identity?.IsAuthenticated == true)
{
var tenantId = context.User.FindFirst("tenant_id")?.Value;
if (!string.IsNullOrEmpty(tenantId))
{
logEvent.AddOrUpdateProperty(propertyFactory.CreateProperty("TenantId", tenantId));
}
}
}
}
// 在Program.cs中注冊
builder.Services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
Log.Logger = new LoggerConfiguration()
.Enrich.With<TenantEnricher>()
// ... 其他配置
.CreateLogger();3. 性能關(guān)鍵的日志記錄
對于高吞吐量場景，使用源生成的日志記錄：
public partialclassOrderService
{
privatereadonly ILogger<OrderService> _logger;
[LoggerMessage(
EventId = 1001,
Level = LogLevel.Information,
Message = "Processing order {OrderId} for user {UserId} with {ItemCount} items totaling {TotalAmount:C}")]
public static partial void LogOrderProcessing(ILogger logger, int orderId, string userId, int itemCount, decimal totalAmount);
[LoggerMessage(
EventId = 1002,
Level = LogLevel.Error,
Message = "Failed to process order {OrderId}: {ErrorReason}")]
public static partial void LogOrderProcessingError(ILogger logger, Exception exception, int orderId, string errorReason);
public async Task ProcessOrderAsync(Order order)
{
LogOrderProcessing(_logger, order.Id, order.UserId, order.Items.Count, order.TotalAmount);
try
{
// 處理訂單...
}
catch (Exception ex)
{
LogOrderProcessingError(_logger, ex, order.Id, ex.Message);
throw;
}
}
}這會生成零分配的日志記錄代碼，以實現(xiàn)最佳性能。
生產(chǎn)環(huán)境最佳實踐
1. 安全和敏感數(shù)據(jù)
切勿記錄敏感信息。使用Serilog的解構(gòu)策略來清理數(shù)據(jù)：
public classSensitiveDataPolicy : IDestructuringPolicy
{
public bool TryDestructure(object value, ILogEventPropertyValueFactory propertyValueFactory, out LogEventPropertyValue result)
{
result = null;
if (valueis CreditCard card)
{
result = propertyValueFactory.CreatePropertyValue(new
{
Last4Digits = card.Number?.Substring(card.Number.Length - 4),
ExpiryMonth = card.ExpiryMonth,
ExpiryYear = card.ExpiryYear
// 切勿記錄完整號碼或CVV
});
returntrue;
}
returnfalse;
}
}
Log.Logger = new LoggerConfiguration()
.Destructure.With<SensitiveDataPolicy>()
// ... 其他配置
.CreateLogger();2. 特定環(huán)境配置
為每個環(huán)境使用不同的日志記錄配置：
public static void ConfigureLogging(WebApplicationBuilder builder)
{
var environment = builder.Environment.EnvironmentName;
var loggerConfig = new LoggerConfiguration()
.ReadFrom.Configuration(builder.Configuration);
if (environment == "Development")
{
loggerConfig
.MinimumLevel.Debug()
.WriteTo.Console(new JsonFormatter());
}
elseif (environment == "Production")
{
loggerConfig
.MinimumLevel.Information()
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.WriteTo.OpenTelemetry(options =>
{
options.Endpoint = builder.Configuration["OpenTelemetry:Endpoint"];
options.Headers = GetAuthHeaders(builder.Configuration);
});
}
Log.Logger = loggerConfig.CreateLogger();
}3. 性能監(jiān)控
監(jiān)控日志記錄性能以避免影響應(yīng)用程序性能：
// 添加用于監(jiān)控日志記錄性能的指標(biāo)
publicclassLoggingMetrics
{
privatereadonly Counter<long> _logEventsCounter;
privatereadonly Histogram<double> _logProcessingDuration;
public LoggingMetrics(IMeterFactory meterFactory)
{
var meter = meterFactory.Create("MyApp.Logging");
_logEventsCounter = meter.CreateCounter<long>("log_events_total");
_logProcessingDuration = meter.CreateHistogram<double>("log_processing_duration_ms");
}
public void RecordLogEvent(LogEventLevel level)
{
_logEventsCounter.Add(1, new KeyValuePair<string, object>("level", level.ToString()));
}
}常見陷阱及如何避免
1. 過度記錄
問題：記錄所有內(nèi)容會導(dǎo)致噪音和成本增加。
解決方案：使用適當(dāng)?shù)娜罩炯墑e并按命名空間配置最低級別：
.MinimumLevel.Information()
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
.MinimumLevel.Override("Microsoft.EntityFrameworkCore", LogEventLevel.Error)2. 阻塞應(yīng)用程序線程
問題：同步日志記錄會降低應(yīng)用程序速度。
解決方案：使用異步接收器和批處理：
.WriteTo.Async(a => a.OpenTelemetry(options =>
{
options.Endpoint = "http://localhost:4317";
options.BatchingOptions = new BatchingOptions
{
BatchSizeLimit = 1000,
Period = TimeSpan.FromSeconds(2)
};
}))3. 缺失關(guān)聯(lián)上下文
問題：跨服務(wù)邊界的日志未正確關(guān)聯(lián)。
解決方案：確保HTTP調(diào)用中的TraceId傳播：
builder.Services.AddHttpClient<ExternalApiClient>(client =>
{
client.BaseAddress = new Uri("https://api.external.com");
})
.AddHttpMessageHandler<CorrelationIdHandler>();
publicclassCorrelationIdHandler : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var activity = Activity.Current;
if (activity != null)
{
request.Headers.Add("X-Correlation-ID", activity.TraceId.ToString());
}
returnawaitbase.SendAsync(request, cancellationToken);
}
}監(jiān)控和告警
在結(jié)構(gòu)化日志上設(shè)置告警：
# Prometheus的示例告警規(guī)則
groups:
-name:application.alerts
rules:
-alert:HighErrorRate
expr:rate(log_events_total{level="Error"}[5m])>0.1
for:2m
labels:
severity:warning
annotations:
summary:"檢測到高錯誤率"
description:"錯誤率為每秒 {{ $value }} 個錯誤"
-alert:DatabaseErrors
expr:increase(log_events_total{level="Error",logger=~".*Repository.*"}[1m])>5
for:1m
labels:
severity:critical
annotations:
summary: "檢測到數(shù)據(jù)庫錯誤激增"結(jié)果：前后對比
方面 | 之前（傳統(tǒng)） | 之后（Serilog + OpenTelemetry） |
調(diào)試時間 | 數(shù)小時的日志搜索 | 幾分鐘的結(jié)構(gòu)化查詢 |
跨服務(wù)追蹤 | 手動關(guān)聯(lián) | 通過TraceId自動關(guān)聯(lián) |
查詢能力 | 文本搜索/grep | 豐富的結(jié)構(gòu)化查詢 |
告警 | 日志量閾值 | 業(yè)務(wù)邏輯告警 |
性能影響 | 可變 | 可預(yù)測且優(yōu)化 |
團(tuán)隊效率 | 個人偵探工作 | 協(xié)作式可觀測性 |
入門清單
? 安裝Serilog和OpenTelemetry包
? 配置具有JSON輸出的結(jié)構(gòu)化日志記錄
? 使用Docker設(shè)置OpenTelemetry Collector
? 為您的業(yè)務(wù)領(lǐng)域添加上下文擴(kuò)展器
? 按環(huán)境配置不同的日志級別
? 實現(xiàn)敏感數(shù)據(jù)過濾
? 設(shè)置基本告警規(guī)則
? 培訓(xùn)團(tuán)隊進(jìn)行結(jié)構(gòu)化查詢
核心要點
Serilog + OpenTelemetry不僅僅是更好的日志記錄——它是一種可觀測性，改變了您理解和調(diào)試.NET應(yīng)用程序的方式。
當(dāng)凌晨3點的警報響起時，您將擁有：
? 可以立即查詢的結(jié)構(gòu)化數(shù)據(jù)
? 跨所有服務(wù)的完整關(guān)聯(lián)
? 講述完整故事的豐富上下文
? 與日志并行的性能指標(biāo)
責(zé)任編輯：武曉燕
來源：
架構(gòu)師老盧
