Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
705526e
Unifying scale logic
alrod Feb 3, 2023
79a0486
Scale API refinements
mathewc Feb 6, 2023
d7bd09e
Adding host options
alrod Feb 10, 2023
49ede7a
Adding hostig config to webjobs sdk
alrod Feb 11, 2023
3e3b1a0
Fix some issues
alrod Feb 14, 2023
8ad30bd
Update to 3.0.37
alrod Feb 15, 2023
c7bf959
Fix VersionSuffix
alrod Feb 16, 2023
476ba91
Scaler API updates
mathewc Feb 16, 2023
c1a8a93
Merge branch 'scaler-apis-mathewc' into scaler-apis
alrod Feb 17, 2023
d6f781d
Fix commnets
alrod Mar 1, 2023
5d77b9a
Remove some APIs
alrod Mar 2, 2023
79d9086
Fix Microsoft.FeatureManagement reference
alrod Mar 2, 2023
e2b08af
AddHostedService for ScaleMonitorService
alrod Mar 3, 2023
3d15a6a
Changing FeatureManagement to IConfiguration
alrod Mar 6, 2023
b59bf70
Fix ScaleManagerTests
alrod Mar 6, 2023
b7479b5
Fix comments
alrod Mar 8, 2023
256602f
Making GetScalersToSample static
alrod Mar 9, 2023
d101ac2
Fix ScaleOptions logging
alrod Mar 9, 2023
8b6936a
Fix typo
alrod Mar 9, 2023
87d1cc2
Fix Vincent comments
alrod Mar 10, 2023
00bf27f
Scaler API updates
mathewc Mar 10, 2023
bd3ebb0
Fix comments
alrod Mar 13, 2023
4467104
Fix comments
alrod Mar 13, 2023
e0c7520
AddOptionsLogging fix
alrod Mar 14, 2023
9f68638
Always create properties in TriggerMetadata
alrod Mar 14, 2023
26ed467
Introducing IScaleStatusProvider
alrod Mar 15, 2023
fcc5aaf
Fix ScaleHostEndToEndTests
alrod Mar 16, 2023
bdf6ac7
Last fixes
alrod Mar 22, 2023
82cd90a
Fix AsyncChainEndToEndTests
alrod Mar 22, 2023
be2550e
Try to fix CI
alrod Mar 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build/common.props
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<Project>
<PropertyGroup>
<!-- Packages can have independent versions and only increment when released -->
<Version>3.0.36$(VersionSuffix)</Version>
<HostStorageVersion>5.0.0-beta.2$(VersionSuffix)</HostStorageVersion>
<Version>3.0.37$(VersionSuffix)</Version>
<HostStorageVersion>5.0.0-beta.3$(VersionSuffix)</HostStorageVersion>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package is about to be published as a GA (non-beta) release, OK to add the changes here?

@alrod alrod Mar 22, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to <HostStorageVersion>5.0.0$(VersionSuffix)</HostStorageVersion>

<LoggingVersion>4.0.3$(VersionSuffix)</LoggingVersion>

<TargetFramework>netstandard2.0</TargetFramework>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,12 @@ public static void AddAzureStorageCoreServices(this IServiceCollection services)
// Adds necessary Azure services to create clients
services.AddAzureClientsCore();

services.TryAddSingleton<IAzureBlobStorageProvider, AzureStorageProvider>();
services.AddAzureStorageScaleServices();
Comment thread
alrod marked this conversation as resolved.
Outdated
}

public static void AddAzureStorageScaleServices(this IServiceCollection services)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to ever be called by a customer?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this will be called from ScaleController, see example here:
https://github.com/Azure/azure-webjobs-sdk/blob/scaler-apis/test/Microsoft.Azure.WebJobs.Host.EndToEndTests/Scale/ScaleHostEndToEndTests.cs#L84

Function host is not supposed to call this.

{
services.TryAddSingleton<IAzureBlobStorageProvider, AzureStorageProvider>();
services.AddSingleton<IConcurrencyStatusRepository, BlobStorageConcurrencyStatusRepository>();
Comment thread
chiangvincent marked this conversation as resolved.
}

Expand Down
1 change: 1 addition & 0 deletions src/Microsoft.Azure.WebJobs.Host/Constants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ internal static class Constants
public const string AzureWebsiteInstanceId = "WEBSITE_INSTANCE_ID";
public const string AzureWebsiteContainerName = "CONTAINER_NAME";
public const string DateTimeFormatString = "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fffK";
public const string HostingConfigSectionName = "HostingConfig";
Comment thread
alrod marked this conversation as resolved.
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,17 @@
using System;
using System.Linq;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Azure.WebJobs.Host.Config;
using Microsoft.Azure.WebJobs.Host.Executors;
using Microsoft.Azure.WebJobs.Host.Scale;
using Microsoft.Azure.WebJobs.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.EnvironmentVariables;
using Microsoft.Extensions.Configuration.Json;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.FeatureManagement;

namespace Microsoft.Extensions.Hosting
{
Expand Down Expand Up @@ -135,6 +140,38 @@ public static IHostBuilder ConfigureWebJobs(this IHostBuilder builder, Action<Ho
return builder;
}

/// <summary>
/// Configures the specified <see cref="IHostBuilder"/> as a scale manager host.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that something a customer will do? Or do we need to mention this is only for internal infrastructure or something like that?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this will be called from `ScaleController and a webjobs customer is not suppose to create a "scale manager host". I I extended the description.

/// </summary>
/// <param name="builder">The <see cref="IHostBuilder"/> to configure.</param>
/// <param name="configure">Configuration action to perform as part of service configuration.</param>
/// <param name="configureScaleOptions">Configuration action for <see cref="ScaleOptions"/>.</param>
/// <returns>The <see cref="IHostBuilder"/>.</returns>
public static IHostBuilder ConfigureWebJobsScale(this IHostBuilder builder,
Action<HostBuilderContext, IWebJobsBuilder> configure,
Action<ScaleOptions> configureScaleOptions)
{
builder.ConfigureServices((context, services) =>
{
WebJobsBuilder webJobsBuilder = new WebJobsBuilder(services);
configure?.Invoke(context, webJobsBuilder);

if (configureScaleOptions != null)
{
services.Configure(configureScaleOptions);
}

services.AddFeatureManagement();
services.TryAddSingleton<IScaleMonitorManager, ScaleMonitorManager>();
services.TryAddSingleton<ITargetScalerManager, TargetScalerManager>();
services.TryAddSingleton<IScaleManager, ScaleManager>();
services.TryAddSingleton<IHostedService, ScaleMonitorService>();
Comment thread
alrod marked this conversation as resolved.
Outdated
services.TryAddSingleton<IPrimaryHostStateProvider>(new PrimaryHostStateProvider() { IsPrimary = true });
Comment thread
alrod marked this conversation as resolved.
Outdated
});

return builder;
}

private static IConfigurationBuilder TryAddDefaultConfigurationSources(this IConfigurationBuilder config)
{
if (!config.Sources.OfType<JsonConfigurationSource>().Any(p => string.Equals(p.Path, "appsettings.json", StringComparison.OrdinalIgnoreCase)))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ public static IWebJobsBuilder AddWebJobs(this IServiceCollection services, Actio
services.TryAddSingleton<IDistributedLockManager, InMemoryDistributedLockManager>();
services.TryAddSingleton<IScaleMonitorManager, ScaleMonitorManager>();
services.TryAddSingleton<ITargetScalerManager, TargetScalerManager>();
services.TryAddSingleton<IScaleManager, ScaleManager>();

services.AddSingleton<IPrimaryHostStateProvider, PrimaryHostStateProvider>();
services.AddSingleton<IHostedService, PrimaryHostCoordinator>();
Expand Down
23 changes: 23 additions & 0 deletions src/Microsoft.Azure.WebJobs.Host/Scale/IScaleManager.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;

namespace Microsoft.Azure.WebJobs.Host.Scale
{
/// <summary>
/// Provides scale status.
/// </summary>
public interface IScaleManager
{
/// <summary>
/// Gets the current scale status <see cref="ScaleStatus"> for all monitored functions.
/// </summary>
/// <param name="context">The <see cref="ScaleStatusContext"/>.</param>
/// <returns>The current <see cref="ScaleStatus"/>.</returns>
Task<ScaleStatus> GetScaleStatusAsync(ScaleStatusContext context);
}
}
28 changes: 28 additions & 0 deletions src/Microsoft.Azure.WebJobs.Host/Scale/IScaleMetricsRepository.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using System.Collections.Generic;
using System.Threading.Tasks;

namespace Microsoft.Azure.WebJobs.Host.Scale
{
/// <summary>
/// Interface defining methods for reading/writing scale metrics to a persistent store.
/// </summary>
public interface IScaleMetricsRepository
{
/// <summary>
/// Persist the metrics for each monitor.
/// </summary>
/// <param name="monitorMetrics">The collection of metrics for each monitor.</param>
/// <returns>A task.</returns>
Task WriteMetricsAsync(IDictionary<IScaleMonitor, ScaleMetrics> monitorMetrics);

/// <summary>
/// Read the metrics.
/// </summary>
/// <param name="monitors">The current collection of monitors.</param>
/// <returns>Map of metrics per monitor.</returns>
Task<IDictionary<IScaleMonitor, IList<ScaleMetrics>>> ReadMetricsAsync(IEnumerable<IScaleMonitor> monitors);
}
}
27 changes: 27 additions & 0 deletions src/Microsoft.Azure.WebJobs.Host/Scale/IScalerProvider.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using System;
using System.Collections.Generic;
using System.Text;

namespace Microsoft.Azure.WebJobs.Host.Scale
{
/// <summary>
/// Interface for providing scalers.
/// </summary>
public interface IScalerProvider
Comment thread
alrod marked this conversation as resolved.
Outdated
{
/// <summary>
/// Gets a collection of <see cref="IScaleMonitor"/>s.
/// </summary>
/// <returns>The <see cref="IScaleMonitor"/>s.</returns>
IEnumerable<IScaleMonitor> GetScaleMonitors();

/// <summary>
/// Gets a collection of <see cref="ITargetScaler"/>s.
/// </summary>
/// <returns>The <see cref="ITargetScaler"/>s.</returns>
IEnumerable<ITargetScaler> GetTargetScalers();
}
}
19 changes: 19 additions & 0 deletions src/Microsoft.Azure.WebJobs.Host/Scale/ITriggerMetadataProvider.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using System.Collections.Generic;

namespace Microsoft.Azure.WebJobs.Host.Scale
{
/// <summary>
/// Provides trigger metadata.
/// </summary>
public interface ITriggerMetadataProvider
{
/// <summary>
/// Gets the trigger metadata.
/// </summary>
/// <returns>The trigger metadata.</returns>
IEnumerable<TriggerMetadata> GetTriggerMetadata();
}
}
Loading