Skinned Mesh attributes #441

Squareys · 2020-05-03T14:30:07Z

Hey @mosra !

here's some initial draft for Skinning stuff to start discussion.

Cheers,
Jonathan

Open Questions
1.Naming of Attributes

TODOs

Add Weights and JointIds to AttributeNames and Generic attributes.
MeshData & Testing
Documentation?

mosra

And here you have some models with more than 4 weights: https://twitter.com/zeuxcg/status/1194650202696605697

src/Magnum/Shaders/Generic.h

mosra · 2020-05-03T14:45:14Z

src/Magnum/Trade/MeshData.h

+         * @see @ref jointIndicesInto(), @ref attributeFormat(),
+         *      @ref isVertexFormatImplementationSpecific()
+         */
+        Containers::Array<Vector4us> jointIndicesAsArray(UnsignedInt id = 0) const;


I think should always be the largest type (even if it will never be as big):

Suggested change

Containers::Array<Vector4us> jointIndicesAsArray(UnsignedInt id = 0) const;

Containers::Array<Vector4ui> jointIndicesAsArray(UnsignedInt id = 0) const;

src/Magnum/Shaders/Generic.h

src/Magnum/Trade/MeshData.h

src/Magnum/Shaders/Generic.h

mosra · 2020-05-08T16:51:07Z

What's left for this to be useful is adding those to MeshTools::compile() -- but to test it there, you would need some shader that supports them, and at that point you have implemented skinning completely :)

Squareys · 2020-05-09T08:43:17Z

Sure, it looks straight forward enough to do so, and I'll have to eventually anyways, so why not rip off the bandage.

Squareys · 2020-05-11T12:25:47Z

@mosra I started implementing Skinning in Phong to test the additions to compile(), but this would blow up the PR significantly. Could we split Phong shader implementation, compile() and test off into a separate PR?

mosra · 2020-05-11T12:48:42Z

Yup, good idea -- the Phong implementation also needs tests on its own, so it makes sense to do all that separately.

Signed-off-by: Squareys <[email protected]>

Squareys · 2020-05-11T12:55:41Z

Split Phong implementation into Squareys:phong-skinning, will create the PR for that once this is merged.
Rebased this and squashed some commits, I think this is ready for review :)

mosra · 2020-05-11T14:54:43Z

src/Magnum/Shaders/Generic.h

+     * @ref Magnum::Vector4ui "Vector4ui", four joint indices that may affect the vertex.
+     * Corresponds to @ref Trade::MeshAttribute::JointIds.
+     */
+    typedef GL::Attribute<7, Vector4ui> JointIds;


I don't see the Secondary attribs documented here.

mosra · 2020-05-11T15:00:19Z

src/Magnum/Trade/Test/MeshDataTest.cpp

+    CORRADE_COMPARE(out.str(),
+        "Trade::MeshData::jointIdsInto(): expected a view with 3 elements but got 2\n");
+}
+


The attributeNotFound() and implementationSpecificVertexFormatWrongAccess() tests need to be extended to check asserts in the new accessors.

mosra · 2020-05-11T15:12:02Z

src/Magnum/Trade/MeshData.h

+     * Corresponds to @ref Shaders::Generic::JointIds.
+     * @see @ref MeshData::jointIdsAsArray()
+     */
+    JointIds,


I'm wondering -- would it make sense to support also one/two/three-component weights and joints? I guess it would be useful for file size savings. Can you check if there are example glTF files with less than 4 weights?

This won't complicate the implementation too much, a similar thing is already done for colors for example.

The accessor is always a 4-component vector, so in case there is a file like this, it would have 4 but the unused components would be zero weight

In (unextended) glTF yes, but in general not. I don't see why glTF couldn't support SCALAR, VEC2 and VEC3 also.

So, if it would make sense to support one/two/three-component weights, would I do that with JointIds1/2/3/4? And I assume the respective changes in MeshData like for color3/4?

It would be just JointIds always, but allowing also one/two/three-component types instead of just Vectot4ui.

MeshData::jointIdsInto() will then need to zero-fill the unused components like colorsInto() does for three-component colors or positions3DInto() does for two-component positions, yes.

mosra · 2020-05-11T15:26:42Z

src/Magnum/Shaders/Generic.h

+    typedef GL::Attribute<7, Vector4> JointIds;
+
+    typedef GL::Attribute<10, Vector4> SecondaryWeights;
+    typedef GL::Attribute<11, Vector4> SecondaryJointIds;


Seeing #444, this PR should contain also the generic.glsl changes and corresponding additions to GenericTest::glslMatch()

mosra · 2022-12-15T19:11:54Z

Merged with various changes as 7f94a6f and f447e99. The primary difference is that MeshAttribute::JointIds and Weights is now an array attribute in order to allow a single attribute to have any number of components -- two, four but also for example seven. This allows both for better packing in memory (compared to always having 4 or 8 components) as well as easier processing because with > 4 components the attribute isn't split into two. On the other hand it's still possible to have two JointIds and two Weights for example, and have the second set in a different (e.g., less precise) type.

The builtin shader bindings slots stay four component, with the intention that (a) the shader is either compiled for an appropriate per-vertex component count, (b) a dynamic per-vertex component count is passed to it, or (c) the mesh is repacked to always contain the maximum number of components the shader expects.

A major change with the builtin attribute is that they have a flipped order compared to this PR -- JointIds and SecondaryJointIds are 6 and 10 instead of 7 and 11, Weights and SecondaryWeights are 7 and 11 instead of 6 and 10. This order made more sense to me, i.e. "joint i has a weight j" instead of "weight i is applied to joint j".

mosra marked this pull request as draft May 3, 2020 14:35

mosra changed the title ~~[WIP] Skinned Mesh attributes~~ Skinned Mesh attributes May 3, 2020

mosra reviewed May 3, 2020

View reviewed changes

Squareys force-pushed the skinned-meshes branch from e85941c to 040d170 Compare May 7, 2020 17:25

Squareys mentioned this pull request May 8, 2020

TinyGltfImporter: Import Weights and JointIds mesh attributes mosra/magnum-plugins#84

Closed

4 tasks

mosra reviewed May 8, 2020

View reviewed changes

src/Magnum/Shaders/Generic.h Outdated Show resolved Hide resolved

src/Magnum/Trade/MeshData.h Outdated Show resolved Hide resolved

src/Magnum/Shaders/Generic.h Outdated Show resolved Hide resolved

src/Magnum/Shaders/Generic.h Outdated Show resolved Hide resolved

Squareys force-pushed the skinned-meshes branch from 040d170 to 385ecc9 Compare May 11, 2020 12:24

Squareys added 3 commits May 11, 2020 14:52

Trade: Minor doc fix

fdfa5df

Signed-off-by: Squareys <[email protected]>

Shaders: Add Weights and JointIndices attribute types

52af410

Signed-off-by: Squareys <[email protected]>

Trade: Implement support for Weights & JointIndices in MeshData.

12dd74a

Signed-off-by: Squareys <[email protected]>

Squareys force-pushed the skinned-meshes branch from 385ecc9 to 12dd74a Compare May 11, 2020 12:53

Squareys marked this pull request as ready for review May 11, 2020 12:55

mosra reviewed May 11, 2020

View reviewed changes

Squareys mentioned this pull request May 11, 2020

Phong skinning #444

Closed

mosra reviewed May 11, 2020

View reviewed changes

mosra mentioned this pull request Sep 21, 2020

Complete animation support #101

Open

45 tasks

pezcode mentioned this pull request Jul 3, 2021

AssimpImporter: import skin data mosra/magnum-plugins#99

Closed

pezcode mentioned this pull request Sep 30, 2021

CgltfImporter mosra/magnum-plugins#107

Merged

mosra added this to the 2022.0a milestone Nov 14, 2022

mosra added the changelog mention added label Nov 21, 2022

mosra closed this Dec 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skinned Mesh attributes #441

Skinned Mesh attributes #441

Squareys commented May 3, 2020 •

edited

Loading

mosra left a comment

mosra May 3, 2020

mosra commented May 8, 2020

Squareys commented May 9, 2020

Squareys commented May 11, 2020

mosra commented May 11, 2020

Squareys commented May 11, 2020

mosra May 11, 2020

mosra May 11, 2020

mosra May 11, 2020

Squareys May 12, 2020

mosra May 12, 2020

Squareys Jun 4, 2020

mosra Jun 9, 2020

mosra May 11, 2020

mosra commented Dec 15, 2022

	Containers::Array<Vector4us> jointIndicesAsArray(UnsignedInt id = 0) const;
	Containers::Array<Vector4ui> jointIndicesAsArray(UnsignedInt id = 0) const;

Skinned Mesh attributes #441

Skinned Mesh attributes #441

Conversation

Squareys commented May 3, 2020 • edited Loading

mosra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mosra commented May 8, 2020

Squareys commented May 9, 2020

Squareys commented May 11, 2020

mosra commented May 11, 2020

Squareys commented May 11, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mosra commented Dec 15, 2022

Squareys commented May 3, 2020 •

edited

Loading