fix: Azure Blob transcript key

[infinite-ram]

Fixes #4176

Description

Expecting Azure blob to creat transcript activity and follow the pattern of container/{channelId]/{conversationId}/{Timestamp.ticks}-{activity.id}.json

instead it was creating directory container%2F{channelId}%2F{conversationId}%2F{Timestamp.ticks}-{activid.id}.json due to sanitizeBlobKey method using the encodeURIComponent function.

Specific Changes

• Created optional decodeTranscriptKey boolean property in BlobsTranscriptStoreOptions interface to return a new string representing the decoded version of the given encoded blob transcript key. This remains the default behavior to false but can be overridden by setting decodeTranscriptKey to true.

Steps to reproduce

Prerequisite: Azure Blob

1. Run git clone --branch ramfattah/blobTranscript https://github.com/microsoft/botbuilder-js.git

2. Run yarn and yarn build in the root folder of SDK

3. Open libraries/botbuilder-azure-blobs/tests/blobsTranscriptStore.test.js file

   • Add your blob connectionString and containerName. Note, it will create the container automatically if the name doesn't already exist.
   • 
   • Flip decodeTranscriptKey flag to true
   • 

4. In terminal, run mocha <path-to-blobsTranscriptStore.test.js

   • Ex: mocha /Users/ram/Documents/bot/botbuilder-js/libraries/botbuilder-azure-blobs/tests/blobsTranscriptStore.test.js

5. This will create blob transcript key in the container as the following pattern container/{channelId]/{conversationId}/{Timestamp.ticks}-{activity.id}.json

Before

After

[coveralls]

Pull Request Test Coverage Report for Build 2165663337

• 3 of 6 (50.0%) changed or added relevant lines in 2 files are covered.
• 27 unchanged lines in 4 files lost coverage.
• Overall coverage decreased (-0.002%) to 84.482%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libraries/botbuilder-azure-blobs/src/blobsTranscriptStore.ts	2	3	66.67%
libraries/botbuilder-azure-blobs/src/sanitizeBlobKey.ts	1	3	33.33%

Files with Coverage Reduction	New Missed Lines	%
libraries/botbuilder-azure-blobs/src/sanitizeBlobKey.ts	1	4.76%
libraries/botbuilder-ai/src/qnaCardBuilder.ts	2	84.48%
libraries/botbuilder-ai/src/qnamaker-utils/generateAnswerUtils.ts	3	85.44%
libraries/botbuilder-ai/src/qnaMakerDialog.ts	21	85.34%

Totals
Change from base Build 2073932673:	-0.002%
Covered Lines:	19909
Relevant Lines:	22316

---

💛 - Coveralls

[tariromukute]

Hi @tracyboehrer and @ramfattah . I am trying to use the this from the TranscriptLoggerMiddleware. The middleware does not call logActivity with the options object and does not support setting an options object see code here

I think the logActivity in the blobsTranscriptStore should also use this._isDecodeTranscriptKey to set options for getBlobKey. This way, the middleware won't need to maintain the same attribute.

What are your thoughts, can we create a new issue for this?

[tracyboehrer]

@tariromukute The main issue is that what options are being used? Blobs options? The transcript middleware doesn't know anything at all about which Storage is being used (Memory, CosmosDB, Blobs, etc...). It only knows about TranscriptLogger. So while the BlobsTranscriptStore has an optional options argument, it's not known to the middleware.

One option might be to implement a transcript logger that does have some knowledge about the store being used. Though perhaps a better option would be to create a BlobsTranscriptStore using a particular BlobsStorageOptions.

[tariromukute]

Hi @tracyboehrer, I agree with you there, the transcript middleware doesn't know anything about the Storage. I always think it will be reperative work to implement that in the transcript middleware/logger. My suggestion is to update BlobsTranscriptStore so that it uses the configuration that it was created with. Say

// index.js
const blobTranscriptStore = new BlobsTranscriptStore(BlobConnectionString, BlobTranscriptContainerName, { decodeTranscriptKey: true })

adapter.use(new TranscriptLoggerMiddleware(blobTranscriptStore));

We have passed blobTranscriptStore that has the options arguments. The BlobTrascriptStore stores this argument as this._isDecodeTranscriptKey

export class BlobsTranscriptStore implements TranscriptStore {
    private readonly _containerClient: ContainerClient;
    private readonly _concurrency = Infinity;
    private _initializePromise?: Promise<unknown>;
    private _isDecodeTranscriptKey?: boolean = false;

    /**
     * Constructs a BlobsTranscriptStore instance.
     *
     * @param {string} connectionString Azure Blob Storage connection string
     * @param {string} containerName Azure Blob Storage container name
     * @param {BlobsTranscriptStoreOptions} options Other options for BlobsTranscriptStore
     */
    constructor(connectionString: string, containerName: string, options?: BlobsTranscriptStoreOptions) {
        z.object({ connectionString: z.string(), containerName: z.string() }).parse({
            connectionString,
            containerName,
        });

        this._containerClient = new ContainerClient(connectionString, containerName, options?.storagePipelineOptions);

        this._isDecodeTranscriptKey = options?.decodeTranscriptKey;

        // At most one promise at a time to be friendly to local emulator users
        if (connectionString.trim() === 'UseDevelopmentStorage=true;') {
            this._concurrency = 1;
        }
    }

}

My suggestion is we update logActivity in the BlobTrascriptStore.

    /**
     * Log an activity to the transcript.
     *
     * @param {Activity} activity activity to log
     * @param {BlobsTranscriptStoreOptions} options Optional settings for BlobsTranscriptStore
     * @returns {Promise<void>} A promise representing the async operation.
     */
    async logActivity(activity: Activity, options?: BlobsTranscriptStoreOptions): Promise<void> {
        z.object({ activity: z.record(z.unknown()) }).parse({ activity });

        await this._initialize();

        const blob = this._containerClient.getBlockBlobClient(getBlobKey(activity, options));
        const serialized = JSON.stringify(activity);

        const metadata: Record<string, string> = {
            FromId: activity.from.id,
            RecipientId: activity.recipient.id,
        };

        if (activity.id) {
            metadata.Id = activity.id;
        }

        if (activity.timestamp) {
            metadata.Timestamp = activity.timestamp.toJSON();
        }

        await blob.upload(serialized, serialized.length, { metadata });
    }

In the change, we update logActivity such that before calling const blob = this._containerClient.getBlockBlobClient(getBlobKey(activity, options)); we check if this._isDecodeTranscriptKey is true. Something like

       await this._initialize();

        if (this._isDecodeTranscriptKey && options && options.decodeTranscriptKey === undefined) {
            options = { decodeTranscriptKey: true, ...options };
        }
        const blob = this._containerClient.getBlockBlobClient(getBlobKey(activity, options));
        const serialized = JSON.stringify(activity);