在yarn中的application详情页面
http://resourcemanager/cluster/app/$applicationId
或者通过application命令
yarn application -status $applicationId
只能看到应用启动以来占用的资源*时间统计，比如：
Aggregate Resource Allocation : 3962853 MB-seconds, 1466 vcore-seconds
到处都找不到这个应用当前实时的资源占用情况，比如当前占用了多少内存多少核，跟进yarn代码发现其实是有这个统计的：
org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport
public static ApplicationResourceUsageReport newInstance( int numUsedContainers, int numReservedContainers, Resource usedResources, Resource reservedResources, Resource neededResources, long memorySeconds, long vcoreSeconds) { ApplicationResourceUsageReport report = Records.newRecord(ApplicationResourceUsageReport.class); report.setNumUsedContainers(numUsedContainers); report.setNumReservedContainers(numReservedContainers); report.setUsedResources(usedResources); report.setReservedResources(reservedResources); report.setNeededResources(neededResources); report.setMemorySeconds(memorySeconds); report.setVcoreSeconds(vcoreSeconds); return report; }
其中usedResources就是当前的实时占用资源情况，包括内存和cpu，这个统计是在YarnScheduler的接口中返回：
org.apache.hadoop.yarn.server.resourcemanager.scheduler.YarnScheduler
/** * Get a resource usage report from a given app attempt ID. * @param appAttemptId the id of the application attempt * @return resource usage report for this given attempt */ @LimitedPrivate("yarn") @Evolving ApplicationResourceUsageReport getAppResourceUsageReport( ApplicationAttemptId appAttemptId);
getAppResourceUsageReport方法被RMAppAttemptImpl.getApplicationResourceUsageReport调用：
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl
@Override public ApplicationResourceUsageReport getApplicationResourceUsageReport() { this.readLock.lock(); try { ApplicationResourceUsageReport report = scheduler.getAppResourceUsageReport(this.getAppAttemptId()); if (report == null) { report = RMServerUtils.DUMMY_APPLICATION_RESOURCE_USAGE_REPORT; } AggregateAppResourceUsage resUsage = this.attemptMetrics.getAggregateAppResourceUsage(); report.setMemorySeconds(resUsage.getMemorySeconds()); report.setVcoreSeconds(resUsage.getVcoreSeconds()); return report; } finally { this.readLock.unlock(); } }
RMAppAttemptImpl.getApplicationResourceUsageReport被两个地方调用：
第一个调用org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl
public ApplicationReport createAndGetApplicationReport(String clientUserName, boolean allowAccess) { ... appUsageReport = currentAttempt.getApplicationResourceUsageReport(); ...
RMAppImpl.createAndGetApplicationReport会被ClientRMService.getApplications和ClientRMService.getApplicationReport调用，这两个方法分别对应命令
yarn application -list
yarn application -status $applicationId
这两个地方展示信息的时候都没展示usedResources，可能作者觉得这个实时资源占用统计没那么重要。
详见：
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService
org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo
public AppInfo(RMApp app, Boolean hasAccess, String schemePrefix) { ... ApplicationResourceUsageReport resourceReport = attempt .getApplicationResourceUsageReport(); if (resourceReport != null) { Resource usedResources = resourceReport.getUsedResources(); allocatedMB = usedResources.getMemory(); allocatedVCores = usedResources.getVirtualCores(); runningContainers = resourceReport.getNumUsedContainers(); } ...
这个构造函数会在RMWebServices.getApp和RMWebServices.getApps时被调用，这是个service接口，对应url分别为：
http://resourcemanager/ws/v1/cluster/apps/$applicationId
http://resourcemanager/ws/v1/cluster/apps?state=RUNNING
这两个接口的返回值中有实时资源占用情况如下：
<allocatedMB>56320</allocatedMB>
<allocatedVCores>21</allocatedVCores>
分别对应实时内存占用和实时CPU占用；
详见：
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices
如果你发现spark应用内存的占用比你分配的要多，可以参考这里：https://www.cnblogs.com/barneywill/p/10102353.html