Microsoft 365 usage reports – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

The Microsoft 365 usage reports are available inside the Microsoft 365 admin center. They are broad reports that can be used to get a high-level snapshot of how your organization uses the Microsoft 365 platform. Report data includes statistics about how many files are stored in SharePoint, how many Exchange mailboxes were active during the reporting period, as well as engagement with other products such as Yammer or Forms:

Figure 2.21 – Microsoft 365 usage reports

Usage reports can be accessed by navigating to the Microsoft 365 admin center (https://admin.microsoft.com), expanding Reports, and selecting Usage.

Viva Insights

Formerly known as Workplace Analytics, Viva Insights provides recommendations about personal and teamwork habits. Viva Insights has four core areas:

  • Personal insights
  • Teamwork habits
  • Organization trends
  • Advanced insights

Each of these areas has unique features that are part of the Viva story.

Personal insights

As the name suggests, personal insights are tailored to an individual. Personal insights are private and are only visible to the individual for whom they are intended. Personal insights are best viewed using the Viva Insights app in Microsoft Teams, as shown in Figure 2.22:

Figure 2.22 – Viva Insights app in Microsoft Teams

The Viva Insights app has functions to allow you to make a focus plan (sometimes referred to as the protect time feature), send praise to your colleagues either publicly or privately, and stay connected through AI-based task suggestions and meeting assistance.

The Viva Insights app also features Headspace guided meditation and mindfulness exercises, as well as prompts for taking a break and reflecting on your personal feelings. Using the Reflection activity card, you can even set daily reminders to check in on yourself:

Figure 2.23 – Reflection activity card

Viva Insights also has a daily ramp-up and wind-down micro-app called Virtual commute, which lets users review upcoming meetings and tasks, block focus time, and initiate a variety of mini-break, meditative, and reflective activities. See Figure 2.23:

Figure 2.24 – Virtual commute activity card

Together, these insights features can help users manage both their productivity and personal well-being.

Sign-in logs – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

The Sign-ins activity report provides data regarding sign-in activity for your tenant, including users and other security or service principals. The report includes information regarding the user, the status of the request, the resource name used for the sign-in, whether multi-factor authentication or conditional access was required, as well as regional location and IP address information:

Figure 2.17 – Sign-in logs

Selecting an individual sign-in event brings up advanced details. Each tab contains additional information regarding the sign-in event. See Figure 2.18:

Figure 2.18 – Sign-in activity details

Sign-ins logs are available to all subscriptions, though programmatic access to this data via the Graph API requires either Azure AD Premium P1 or P2.

Provisioning logs

The provisioning logs show data regarding users being provisioned into Azure AD from connected applications or to connected applications from Azure AD provisioning workflows.

To view the provisioning logs, a user must be granted one of the following roles:

  • Reports Reader
  • Security Reader
  • Security Operator
  • Security Administrator
  • Application Administrator
  • Cloud Application Administrator
  • Global Administrator

Objects created manually through the Azure AD portal, PowerShell, or Microsoft 365 admin center do not appear here, nor do objects synchronized via Azure AD Connect.

Azure Monitor and Log Analytics

Azure Monitor provides a single, unified hub for diagnostic and monitoring data in Azure and connected applications. The easiest way to start reviewing the logs is to select the Log Analytics link under the Monitoring section in Azure Active Directory, as shown in Figure 2.19:

Figure 2.19 – Accessing Log Analytics from the Monitoring section of Azure AD

Log analytics data can be searched using built-in queries or by specifying your own searches in the Query window.

For example, you can select built-in queries to begin querying data immediately. Figure 2.20 shows a query for the SigninLogs table, summarizing sign-ins by country:

Figure 2.20 – Querying Log Analytics

Deep-dive into Kusto Query Language

Kusto Query Language (KQL) is used to search for and sort through data in Log Analytics. It is an incredibly powerful language but takes some time to learn. KQL is used in Log Analytics, Azure Monitor, and Azure Sentinel. If you want to start learning KQL, you can work through the Log Analytics tutorial at https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-tutorial.

Reviewing usage metrics

For your organization to get the most benefit from a Microsoft 365 investment, users must adopt the available services and features. You can monitor end user adoption and consumption metrics through a variety of tools, including Microsoft 365 Usage Metrics, Viva Insights (formerly known as Workplace Analytics), and Adoption Score (formerly known as Productivity Score).

Connecting to Azure Monitor – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

If you have an Azure subscription with a Log Analytics workspace created and at least Azure AD Premium P1, you can send Azure Active Directory activity log data to Azure Monitor easily by following these steps:

  1. From the Azure portal (https://portal.azure.com), navigate to Azure Active Directory.
  2. Under Monitoring, select Diagnostic settings and then click + Add diagnostic setting:

Figure 2.15 – Configuring Azure AD diagnostic settings

3. Under Logs, select one or more categories of logs to send to the workspace.

4. Under Destination details, check the Send to Log Analytics workspace checkbox and then select an Azure Subscription and Log Analytics workspace. Click Save when you have finished selecting these options:

Figure 2.16 – Selecting diagnostics settings for Azure Monitor

After about 15 minutes, new logging event data should begin showing up in the Log Analytics workspace.

Configuring and reviewing reports

With reporting data now flowing into Azure Monitor and Log Analytics, you can review auditing and logging data to gain insights into how your tenant and directory services are being used.

To review this data, you’ll need to have access to the Log Analytics workspace where Azure Monitor is sending data, as well as one of the following roles:

  • Global Admin
  • Reports Reader
  • Security Admin
  • Security Reader

With that, let’s start looking at logs!

Azure AD logs and reports

Azure AD provides several default reports that can be used to identify issues quickly. The core reports are the Audit, Sign-in, and Provisioning logs.

Audit log data can be held for up to 10 years, depending on the license:

  • Office 365 E1 or E3; Microsoft 365 F1 or E3: 90 days
  • Office 365 E5; Microsoft 365 E5: 1 year
  • Audit Premium: 10 years

Advanced licensing

For more information on the variety of SKU mixes for audit retention, see https://learn.microsoft.com/en-us/microsoft-365/compliance/audit-solutions-overview?source=recommendations&view=o365-worldwide.

Accessing the audit log data does not require specific licensing, though you will only see audit events for products that you have currently licensed.

Reviewing audit logs – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

Application audit logs are useful for reviewing actions that have occurred in your tenant. You can view these audit logs from the Enterprise applications page in the Azure portal, as shown in Figure 2.13:

Figure 2.13 – Enterprise application audit logs

These audit logs show data regarding the service principal, applications, and type of action performed. You can select an individual audit item to view additional details. You can also perform filtering on several fields such as Status (Success or Failure), Initiated by (actor) (user or security principal that executed the action), User agent (device type or browser where the action was submitted), and Target (application or service that was affected).

You can also select an individual application (Enterprise applications | All applications) and view all of the audit logs that pertain specifically to that application.

Reviewing the sign-ins report

The sign-ins report shows data related specifically to sign-ins. Like the audit log data, you can view it across all applications (from the Enterprise applications | Audit logs page) or just for an individual application, as shown in Figure 2.14:

Figure 2.14 – Application sign-in logs

The application sign-in logs are useful for identifying potentially anomalous or malicious behavior. For example, if you see several failures for a particular user and they have a multifactor authentication requirement configured, the user may have a compromised password. If you see several failures for different users that are related to the same application, you may have an identity provisioning or single sign-on problem that needs to be addressed.

Sending activity log data to Azure Monitor

Azure Monitor is an additional subscription service that can be used to store and analyze logging and auditing data from a variety of sources, including Azure Active Directory, virtual machines, and applications. By connecting Azure AD data to Azure Monitor, you can enable Microsoft Defender to gain access to this data so that you can compare it against security logs, thereby improving risk management:

  1. From the Azure portal (https://portal.azure.com), enter Log Analytics workspaces in the search box.
  2. Click Create.
  3. From the Subscription drop-down menu, select an Azure subscription.
  4. From the Resource group drop-down menu, select an existing resource group or click Create new to create a new one.
  5. Enter a new workspace Name and select a Region option for where you want to provision the workspace.
  6. Click Review + Create.
  7. Click Create.

Once the workspace has been provisioned, you can connect the activity log to Azure Monitor.

Monitoring application access – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

While many cloud-based applications and services may use their own identity stores, it is becoming more common for application vendors to allow bring-you-own-identity scenarios. You might see this with websites allowing social media logins or other types of identity.

Like other identity providers, Azure AD identity can be used to authenticate users to external applications. While many of those applications are legitimate (and their use derives from a legitimate business use case), malicious websites or individuals can publish applications to steal data. As part of your operational practices, you should periodically review allowed applications in your environment and remove the authorizations for applications that look suspicious or are no longer being used.

Applications that are registered or authorized in Azure AD can be used to provide single sign-on to both SaaS cloud applications as well as internally managed applications. Depending on your organization’s settings, applications may be authorized by end users, administrators, or both.

There are several things you can do to monitor application access:

  • Create and manage access reviews
  • Review audit logs
  • Review the sign-ins report
  • Send activity log data to Azure Monitor

Let’s look at each of these areas briefly.

Creating and managing access reviews

The primary goal of an access review is to confirm that those who have access to an application or other resource still have access. If a user, whether internal or external, no longer requires the ability to use a resource, their access to that resource should be terminated.

Note

Access reviews are a feature of Identity Governance and require Azure AD Premium P2.

To create an access review, follow these steps:

  1. Log in to the Azure portal (https://portal.azure.com) with a user that has one of the prerequisite role assignments (Global Administrator, User Administrator, Identity Governance Administrator, or Privileged Role Administrator) or who is an owner of the group for which the access review will be created.
  2. In the search box, enter Identity Governance and select the Identity Governance item.
  3. Under the Access reviews navigation menu item, select Access reviews:

Figure 2.9 – Access reviews

  • Select + New access review.
  • In the Select Review dropdown, select Teams + Groups or Applications:

Figure 2.10 – New access review – the Select Review dropdown

  • Depending on your selection, choose from All Microsoft 365 groups with guest users, Select Teams + groups (if you choose the Teams + Groups option), or one or more registered applications (if you choose the Applications option). If you select Teams + Groups, you may have additional selections regarding specific groups to include or exclude or specific scopes of users to include or exclude.
  • Click Next.
  • Under Specify reviewers, select the individuals who will be responsible for auditing the group. You may be asked to provide Fallback reviewers (if the ones you initially select cease to exist in the future), depending on the options you select.
  • Depending on your settings, you may see an option to perform a multi-stage review. Multi-stage reviews allow you to add up to three stages of reviewers to audit the membership of a group.
  • Under Specify recurrence of review, set a Duration (in days) period, a Review recurrence option (one-time, weekly, monthly, quarterly, semi-annually, or annually), and start and end date parameters. Click Next.
  • Under Upon completion settings, choose whether to Auto apply results to a resource and what to do If reviewers don’t respond:

Figure 2.11 – Access review additional settings

  1. You can also choose to Enable reviewer decision helpers, which are like tooltips that provide additional information on the selected actions during the access review.
  2. Under Advanced settings, you can choose additional options such as Justification required, enable Email notifications and Reminders to complete access reviews, and use a text box to specify Additional context for reviewer email, which can be used to further explain the access review process to the individuals you’ve selected.
  3. Click Next.
  4. Enter a name for the access review, review the configured options, and then click Create to create your access review.

After an access review has been created, Azure AD will evaluate whether it needs to run. If the workflow determines it is time for the access review to run, it will do so.

You can view the status of an access review by clicking on it on the Identity governance | Access reviews page.

Users who have been selected to be reviewers will receive an email notification with a link to the access review page. You can also view the access review by selecting Results under the Manage menu item. From there, you’ll be able to view the recommended actions and the audit information for review:

Figure 2.12 – Access review results

Next, we’ll look at some of the logging and reporting data available for applications.

Creating an incident response plan – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

If an incident occurs that affects the availability of services or features in your tenant, you need to be able to respond quickly. An incident response plan is a framework that you can prepare to help you address issues quickly.

While the details of each incident may differ, the steps you take to both prepare and work through one are the same:

  1. Validate the incident scope details and confirm that your environment is affected. Not all incidents affect all tenants, so use the information in the Message Center (https://admin.microsoft.com/#/MessageCenter), as well as investigative procedures such as self-assessments and tests or synthetic transactions.
  2. Determine whether the incident is relevant to your organization. If the incident involves a service that your organization hasn’t yet deployed or doesn’t interfere with business operations, it may not be relevant.
  3. Once degradation and relevancy to your environment have been confirmed, review information sources for details on the timeline of Microsoft’s response. Microsoft will post regular status updates in the Message Center. If information such as a timeline has not been established, you can open a service ticket with Microsoft to request this information.
  4. Develop a backup solution in case the service outage or degradation lasts longer than an acceptable time frame for your organization. Depending on the type of outage, this may mean working offline to fulfill business requirements.

Business continuity planning (BCP) is important regardless of the technology platforms or services being used. Work with various business unit owners to establish communication plans and methods to continue business operations should a service interruption occur.

Monitoring service health

Service health information is available from the Microsoft 365 admin center (https://admin.microsoft.com). Microsoft provides health information for a variety of services and features, including the SaaS services such as Exchange Online or SharePoint Online, the health of the directory synchronization environment, as well as the Windows operating system feature issues and service health.

You can check the overall service health by navigating to the health dashboard (Health | Dashboard), as shown in Figure 2.5:

Figure 2.5 – Service health dashboard

The health dashboard contains the current health status of all Microsoft 365 services. Normally, services will appear as Healthy, though this status will be updated when a service experiences an issue.

The Service health page (Health | Service health) will display the most detailed and comprehensive information on any ongoing or resolved issues:

Figure 2.6 – Service health page

If a service has an advisory or incident, you can expand the issue item under Active issues to display relevant events, as shown in Figure 2.7:

Figure 2.7 – Service health active issues

Selecting an individual item reveals expanded information about the particular issue. See Figure 2.8 for an example:

Figure 2.8 – Expanded active issue

Each service with an incident will display a status. Possible statuses include the following:

  • Normal service: This status indicates the service is available and has no current incidents or incidents during the reporting period.
  • Extended recovery: This status indicates that while steps have been completed to resolve the incident, it may take time for operations to return to normal. During an extended recovery period, some service operations might be delayed or take longer to complete.
  • Investigating: This status indicates that a potential service incident is being reviewed.
  • Service restored: This status indicates that an incident was active earlier in the day but the service was restored.
  • Service interruption: This status indicates the service isn’t functioning and that affected users are unable to access the service.
  • Additional information: This status indicates the presence of information regarding a recent incident from the previous day.
  • Service degradation: This status indicates that the service is slow or occasionally seems to be unresponsive for brief periods.
  • PIR published: This status indicates that a Post-Incident Report (PIR) of the service incident has been published.
  • Restoring service: This status indicates that the service incident is being resolved.

As an administrator, it’s important to frequently check the Service health dashboard to be apprised of alerts or incidents. If a service issue is affecting the Microsoft 365 admin center, you can also try the Office 365 status page (https://status.office.com) and the Azure status page (https://status.azure.com).

Creating and managing service requests – Monitoring Microsoft 365 Tenant Health

Posted on by Adriana Guarin

As you learned in the previous chapter, the Microsoft 365 tenant is the security and content boundary for your organization. You have control over the content that goes into your tenant, as well as the security controls that you apply to it. However, you don’t have control over external factors such as connectivity between platform services, errors that are introduced from the service provider’s side, or errors with your environment connecting to Microsoft 365.

That’s where monitoring the health of the tenant comes into play.

Like any other service that you are responsible for managing, you also need to be able to develop and execute an incident response plan.

In this chapter, we’ll discuss monitoring and managing the health of a Microsoft 365 tenant. The objectives and skills we’ll cover in this chapter include the following:

  • Creating and managing service requests
  • Creating an incident response plan
  • Monitoring service health
  • Monitoring application access
  • Configuring and reviewing reports
  • Reviewing usage metrics

By the end of this chapter, you should be able to describe day-to-day operations such as monitoring and reporting, as well as important tasks such as creating an incident response plan.

Let’s begin!

Creating and managing service requests

While Microsoft is committed to ensuring the Microsoft 365 platform is as reliable as possible, service interruptions may occur.

Service requests for Microsoft 365 issues are typically raised through the Microsoft 365 admin center. You can create a support request by performing the following steps:

  1. Log in to the Microsoft 365 admin center (https://admin.microsoft.com) and navigate to Support | New service request:

Figure 2.1 – Creating a service request in the Microsoft 365 admin center

  • In the fly-out panel, type in a question or keywords that relate to your service issue or request. If applicable, a list of potential suggested solutions will be displayed. If no suitable options are displayed, you can select Contact Support:

Figure 2.2 – Microsoft 365 service ticket suggestions

  • On the Contact support view, you can fill out any required information, select the preferred option to be contacted, and, once ready, click Contact me:

Figure 2.3 – Contacting support

  • Once a support request has been created, you can select the Support | View service requests option in the Microsoft 365 admin center to track the status of your service request or update it with new information:

Figure 2.4 – Service request history

Summary – Planning and Implementing a Microsoft 365 Tenant

Posted on by Adriana Guarin

In this chapter, you learned about the fundamental aspects and terminology of configuring a Microsoft 365 tenant, such as selecting a tenant and subscription type, adding domains, and configuring the basic organization settings.

In the next chapter, we will learn how to monitor the Microsoft 365 tenant’s health.

Knowledge check

In this section, we’ll test your knowledge of some key elements from this chapter.

Questions

  1. What is the maximum number of domains that can be added to a Microsoft 365 tenant?
    1. 100
    1. 500
    1. 900
    1. 1,000
  2. You are the administrator for an organization with 250 employees. Which Office 365 subscription best fits the size of the organization?
    1. Microsoft 365 Family
    1. Microsoft 365 Business
    1. Microsoft 365 Enterprise
    1. Microsoft 365 Education
  3. You recently took over the administration duties for a Microsoft 365 tenant for a start-up organization. The organization purchased a domain from a third-party registrar. Can this domain be used with Microsoft 365?
    1. Yes
    1. Yes, but it must be transferred to Microsoft first
    1. No
    1. Only domains purchased through the Microsoft 365 admin center can be configured for use with Microsoft 365
  4. Your organization wants to turn off Microsoft Books for all employees until the support staff has had time to read the documentation. From the available options, what should you do?
    1. Disable all Azure AD user accounts
    1. Disable directory synchronization
    1. Disable bookings from Org settings | Services
    1. Disable bookings from Org settings | Security & privacy
  5. The Service Desk manager for Contoso has asked you to update the help desk information for your Microsoft 365 tenant with the internal help desk contact information. Where would you make this update?
    1. Org settings | Organization profile
    1. Org settings | Services
    1. Microsoft Service Now Admin center
    1. Microsoft 365 portal | Account settings

Answers

  1. C: 900
  2. B: Microsoft 365 Business
  3. A: Yes
  4. C: Disable Bookings from Org settings | Services
  5. A: Org settings | Organization profile

Creating a tenant – Planning and Implementing a Microsoft 365 Tenant

Posted on by Adriana Guarin

The act of creating a tenant is a relatively simple affair, requiring you to fill out a basic contact form and choose a tenant name. Microsoft periodically changes what plans are available for new trial subscriptions. As of this writing, Office 365 E3 is available for a trial subscription. Currently available public trial subscriptions require the addition of payment information, which will cause a trial to roll over to a fully-paid subscription after the trial period ends. See Figure 1.2:

Figure 1.2 – Starting a trial subscription

The signup process may prompt for a phone number to be used during verification (either a text/SMS or call) to help ensure that you’re a valid potential customer and not an automated system.

After verifying your status as a human, you’ll be prompted to select your managed domain, as shown in Figure 1.3:

Figure 1.3 – Choosing a managed domain

In the Domain name field, you’ll be prompted to enter a domain name. If the domain name value you select is already taken, you’ll receive an error and be prompted to select a new name.

After you’ve finished, you can enter payment information for a trial subscription. Note the end date of the trial; if you fail to cancel by that time, you’ll be automatically billed for the number of licenses you have configured during your trial!

Implementing and managing domains

The managed domain is part of the Microsoft 365 tenant for its entire lifecycle. While it is a fully-functioning domain name space (complete with its own managed publicly available domain name system), most organizations will want to use their organization’s domain names—especially when it comes to sending and receiving email or communicating via Microsoft Teams.

Organizations can use any public domain name with Microsoft 365. Microsoft supports configuring up to 900 domains in a tenant; you can configure both top-level domains (such as contoso.com) and subdomains (businessunit.contoso.com) with your Microsoft 365 tenant.

Acquiring a domain name

Many organizations begin their Microsoft 365 journey with existing domain names. Those existing domain names can be used with Microsoft 365. In addition, you can purchase new domain names to be associated with your tenant.

Third-party registrar

Most large organizations have existing relationships with third-party domain registrars, such as Network Solutions or GoDaddy. You can use any ICANN-accredited registrar for your region to purchase domain names.

Services – Planning and Implementing a Microsoft 365 Tenant-2

Posted on by Adriana Guarin

While there are no deep questions about what each of the service options do, we recommend you spend time exploring the options for the services in the Microsoft 365 admin center.

Security & privacy

The Security & privacy tab houses settings that govern various security controls for the organization. On this page, you’ll find access to the following settings:


                                                                                 

Setting


                                                                                 

Description


                                                                                 

Bing data collection


                                                                                 

Choose whether to allow Bing to collect organization query data.


                                                                                 

Idle session timeout


                                                                                 

Configure the idle session timeout period for Office web apps.


                                                                                 

Password expiration policy


                                                                                 

Choose whether to enable password expiration. Password expiration is disabled by default (and the password policy is governed by the on-premises Active Directory if password hash sync has been configured).


                                                                                 

Privacy profile


                                                                                 

Configure the URL for the organization’s privacy policy and the organization’s privacy contact. The privacy URL is displayed on the
Privacy
 tab of the
Settings & Privacy
 page in the user account profile and when a sharing request is sent to an external user.


                                                                                 

Self-service password reset


                                                                                 

Provides a link to the Azure portal to configure self-service password reset.


                                                                                 

Sharing


                                                                                 

Choose whether to allow users to add guests to the organization.

Table 1.3 – Security & privacy settings

These options can be used to broadly configure security and privacy settings for your organization. As with the settings on the Services tab, these are coarse controls. Fine-grained control is available for some of these items inside their respective admin centers.

Organization profile

Settings on the Organization profile tab are largely informational or used to manage certain aspects of the user experience. On this tab, you’ll find the following settings:


                                                                                 

Setting


                                                                                 

Description


                                                                                 

Custom app launcher tiles


                                                                                 

Configure additional tiles to show up on the Microsoft 365 app launcher.


                                                                                 

Custom themes


                                                                                 

Create and apply themes to the Microsoft 365 portal for end users, including mandating the theme as well as specific organization logos and colors.


                                                                                 

Data location


                                                                                 

View the regional information where your tenants’ data is stored.


                                                                                 

Help desk information


                                                                                 

Choose whether to add custom help desk support information for end users to the Office 365 help pane.


                                                                                 

Keyboard shortcuts


                                                                                 

View the shortcuts available for use in the Microsoft 365 admin center.


                                                                                 

Organization information


                                                                                 

Update your organization’s name and other contact information.


                                                                                 

Release preferences


                                                                                 

Choose the release settings for Office 365 features (excluding Microsoft 365 apps). The available options are Standard release for everyone, Targeted release for everyone, and Targeted release for select users. The default setting is Standard release for everyone.


                                                                                 

Support integration


                                                                                 

Use the settings on this page to configure integration with third-party support tools such as Service Now.

Table 1.4 – Organization profile settings

Like the other Org settings tabs, the settings on this page will be used infrequently—typically when just setting up your tenant and customizing the experience. As with the other Organization profile setting areas, you should spend some time in a test environment navigating the tenant to view these settings and update them to see their effects.