Skip to content

[ML] File data viz fix fields from mappings #77251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 15, 2020
Merged

[ML] File data viz fix fields from mappings #77251

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
edb7d08
[ML] File data viz fix fields from mappings
jgowdyelastic Sep 10, 2020
bbe2f29
adding card count test
jgowdyelastic Sep 14, 2020
b4b2e51
Merge branch 'master' into fix-fields-in-mappings
elasticmachine Sep 14, 2020
088c544
updating test
jgowdyelastic Sep 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .../public/application/datavisualizer/file_based/components/fields_stats/field_stats_card.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export function FieldStatsCard({ field }) {

return (
<EuiPanel hasShadow={false} className="mlFieldDataCard">
<div className="ml-field-data-card">
<div className="ml-field-data-card" data-test-subj="mlPageFileDataVisFieldDataCard">
<div className={`ml-field-title-bar ${type}`}>
<FieldTypeIcon type={type} needsAria={false} />
<div className="field-name" tabIndex="0" aria-label={`${cardTitleAriaLabel.join(', ')}`}>
Expand Down
6 changes: 3 additions & 3 deletions ...s/ml/public/application/datavisualizer/file_based/components/fields_stats/fields_stats.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ function createFields(results) {
timestamp_field: timestampField,
} = results;

if (mappings && fieldStats) {
if (mappings && mappings.properties && fieldStats) {
const fieldNames = getFieldNames(results);

return fieldNames.map((name) => {
if (fieldStats[name] !== undefined) {
const field = { name };
const f = fieldStats[name];
const m = mappings[name];
const m = mappings.properties[name];

// sometimes the timestamp field is not in the mappings, and so our
// collection of fields will be missing a time field with a type of date
Expand Down Expand Up @@ -93,7 +93,7 @@ function createFields(results) {
// this could be the message field for a semi-structured log file or a
// field which the endpoint has not been able to work out any information for
const type =
mappings[name] && mappings[name].type === ML_JOB_FIELD_TYPES.TEXT
mappings.properties[name] && mappings.properties[name].type === ML_JOB_FIELD_TYPES.TEXT
? ML_JOB_FIELD_TYPES.TEXT
: ML_JOB_FIELD_TYPES.UNKNOWN;

Expand Down
2 changes: 1 addition & 1 deletion ...l/public/application/datavisualizer/file_based/components/fields_stats/get_field_names.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export function getFieldNames(results) {
// there may be fields in the mappings which do not exist in the field_stats
// e.g. the message field for a semi-structured log file, as they have no stats.
// add any extra fields to the list
const differenceFields = difference(Object.keys(mappings), tempFields);
const differenceFields = difference(Object.keys(mappings.properties), tempFields);

// except @timestamp
const timestampIndex = differenceFields.indexOf('@timestamp');
Expand Down
4 changes: 4 additions & 0 deletions x-pack/test/functional/apps/ml/data_visualizer/file_data_visualizer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ export default function ({ getService }: FtrProviderContext) {
expected: {
results: {
title: 'artificial_server_log',
numberOfFields: 4,
},
},
},
Expand Down Expand Up @@ -62,6 +63,9 @@ export default function ({ getService }: FtrProviderContext) {
await ml.dataVisualizerFileBased.assertFileContentPanelExists();
await ml.dataVisualizerFileBased.assertSummaryPanelExists();
await ml.dataVisualizerFileBased.assertFileStatsPanelExists();
await ml.dataVisualizerFileBased.assertNumberOfFieldCards(
testData.expected.results.numberOfFields
);

await ml.testExecution.logTestStep('loads the import settings page');
await ml.dataVisualizerFileBased.navigateToFileImport();
Expand Down
7 changes: 7 additions & 0 deletions x-pack/test/functional/services/ml/data_visualizer_file_based.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ export function MachineLearningDataVisualizerFileBasedProvider(
await testSubjects.existOrFail('mlFileDataVisFileStatsPanel');
},

async assertNumberOfFieldCards(number: number) {
const cards = await testSubjects.findAll('mlPageFileDataVisFieldDataCard');
if (cards.length !== number) {
throw new Error(`expected ${number} field cards to exist, but found ${cards.length}`);
Copy link
Member

@pheyos pheyos Sep 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this works, I'd suggest to use expect which is used for most of our assertions, something like

expect(cards.length).to.eql(number, `expected ${number} field cards to exist, but found ${cards.length}`);

Copy link
Member Author

@jgowdyelastic jgowdyelastic Sep 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed in 088c544

}
},

async assertImportButtonEnabled(expectedValue: boolean) {
const isEnabled = await testSubjects.isEnabled('mlFileDataVisOpenImportPageButton');
expect(isEnabled).to.eql(
Expand Down