# Lesson 3: Scaling individual segments based on subject-specific data from medical images

This tutorial presumes that you have completed Scaling tutorial Lesson 1: Joint to joint scaling methods andScaling tutorial Lesson 2: Scaling based on external body measurements.

This lesson introduces an advanced approach to
scaling based on a sequence of affine and non-affine
transformations. Each of these transforms is constructed
based either on subject-specific geometry or
on a set of landmarks selected on the bone
surface. As opposed to Lessons 1 and 2, this lesson is
rather methodological than conceptual and provides a good
overview of how to pipeline and combine different 3D
transforms to obtain subject-specific morphing and
registration between frames of reference.

## Linear point- based scaling

Previously described scaling schemes are based on anthropometric measurements and affine transform scaling. Such schemes are good assumptions when more accurate measurements are not feasible or not available. Therefore, these schemes are used quite often. However, a natural next step would be to improve the precision of a model by utilizing available medical images. These can be used to reconstruct subject-specific geometry used for modeling purposes. Medical images contain more detailed information about the bone shape and local deformities that cannot be handled by the anthropometric regression equations.

The simplest inclusion of the subject-specific bone shape from medical image data is to find the affine (linear) transformation that fits a number of corresponding points that are selected on the source and the target geometries. These points could be fitted e.g. in a least-squares manner. This approach is similar to utilizing external body measurements as it relies on a linear transform. However, it is less dependent on the bone orientation and prior knowledge of dimensions to be measured. For example, you can locate any two points on source and target surface consistently without thinking of how a segment length was measured.

Let us make a simple example of using landmark-based affine scaling. First, please download two femur surfaces, SourceFemur.stl and TargetFemur.stl and save them in your working directory. These femur geometries will be used for the rest of this tutorial. The source surface is an unscaled femur used in the standard AnyBody™ models in the AMMR. The target surface is a femur reconstructed from a CT image and saved as a surface mesh in STL format (courtesy of Prof. Sebastian Dendorfer, University of Regensburg, Germany).

Next, please download the AnyScript™ file lesson3a.main.any . This file contains a model with two segments which contain the definition of a surface each, one for the source and one for the target bone. When we load this model, the Model View should show the following picture:

To define a new scaling law let us insert a new AnyFunTransform3DLin2 object after the target segment:

```
AnyFunTransform3DLin2 <ObjectName> =
{
//PreTransforms = {};
Points0 = ;
Points1 = ;
//Mode = VTK_LANDMARK_RIGIDBODY;
};
```

The AnyFunTransform3DLin2 object allows us to build a
transform that fits a set of source and target landmarks in
a least-squares manner as mentioned before. The object
constructs a linear transforms in a full affine (linear
transformation with translation, rotation, size-scaling and
skewing, i.e. 12 degrees of freedom), uniform (orthogonal
rotation with uniform scaling and translation, i.e., 9
d.o.f.), or rigid-body manner (orthogonal rotation of
unscaled object with translation, i.e., 6 d.o.f.). Please
note that the AnyFunTransform3DLin2 object utilizes the
vtk-function/filter *vtkLandmarkTransform*, and,
therefore inherits its modes:

- VTK_LANDMARK_AFFINE
- VTK_LANDMARK_SIMILARITY
- VTK_LANDMARK_RIGIDBODY

A description of this function can be found here.

For this example we want to fit one surface into the other by using a full affine transform. Therefore, we select several corresponding points on the surfaces and put them into the two point-sets called Points0 and Points1, which are the source and target points, respectively. As next step, we change the mode of the AnyFunTransform3DLin2 object to VFK_LANDMARK_AFFINE to use the affine transform:

```
AnyFunTransform3DLin2 MyTransform =
{
//PreTransforms = {};
Points0 =
{{0.0138, 0.0014, 0.0274}, // fovea capitis
{0.0791,-0.3971,-0.0524}, // lateral anterior condyle
{0.094, -0.3954,-0.0183}, // medial anterior condyle
{0.0381,-0.1886,-0.0388}, // anterior mid shaft
{0.0188,-0.3965,-0.0205}, // lateral posterior condyle
{0.0369,-0.3937, 0.0267}}; // medial posterior condyle
Points1 =
{{0.2899, 0.4205, 0.0139}, // fovea capitis
{0.3220, 0.4332,-0.3786}, // lateral anterior condyle
{0.2893, 0.4268,-0.373}, // medial anterior condyle
{0.3289, 0.4259,-0.175}, // anterior mid shaft
{0.3063, 0.4872,-0.3703}, // lateral posterior condyle
{0.2619, 0.4759,-0.3729}}; // medial posterior condyle
Mode = VTK_LANDMARK_AFFINE;
};
```

The selected points on the surface represent specific anatomical landmarks and points described in the comments of the AnyScript™ code. Final modification before we can use the constructed linear transform is to give this transformation a name and apply it to the source surface:

```
AnySeg SourceFemur =
{
Mass = 0; Jii = {0, 0, 0};
AnyDrawSurf Surface =
{
FileName = "SourceFemur.stl";
AnyFunTransform3D &ref = ..MyTransform;
};
};
```

Reloading the model and looking at the bones shown in the Model View, we can see that these bones are now merged. To highlight the differences, we select one of them. This will produce the following picture.

We can see that the source bone is now scaled, i.e., moved,
scaled and skewed to match the target bone. To make that
clear, let us add a new AnyFunTransform3DLin2 called
MyTransform2 to the model which we place after
MyTransform. We want this new transform to perform the
rigid-body registration between target and source surface.
Please note, that we therefore exchange the roles of the
sets of source points Points0 and target points Points1 and
set the mode to VTK_LANDMARK_RIGIDBODY.

Additional to that, we also add a collective transform that contains forward affine and back registration transforms:

```
AnyFunTransform3DLin2 MyTransform2 = {
Points0 = .MyTransform.Points1;
Points1 = .MyTransform.Points0;
Mode = VTK_LANDMARK_RIGIDBODY;
};
AnyFunTransform3DIdentity MyTransform3 =
{
PreTransforms = {&.MyTransform,&.MyTransform2};
};
```

Finally, let us look at the effect of the constructed
transform. We comment the transform used in the
visualization of the source surface and create another
surface that will show the collective transformation that
we just constructed:

```
AnySeg SourceFemur =
{
Mass = 0; Jii = {0, 0, 0};
AnyDrawSurf Surface =
{
FileName = "SourceFemur.stl";
//AnyFunTransform3D &ref = ..MyTransform;
};
AnyDrawSurf SurfaceMorphed =
{
FileName = "SourceFemur.stl";
AnyFunTransform3D &ref = ..MyTransform3;
};
};
```

Looking at the Model View, we can see that
the femur is now scaled, it became shorter and the
alignment match the original source position well. From the
previous picture, we know that geometry is fitting the
target well too (and if you want to convince yourself you
can enter the target geometry too and apply the
MyTransform2 to obtain the registration).

With this example, we have shown how to morph the source into the target with a full affine scaling and subsequently applying a reverse registration to move the morphed geometry back.

Notice that it is possible to reverse the combination, i.e., to apply the registration step first and then the scaling/morphing step. For instance, make a transformation like MyTransform, but insert MyTransform2 as pre-transformation. In this tutorial lesson, we shall however stay with the concept we presented so far.

If the accuracy of the results is sufficient, we could continue working with this simple morphing. However, this transformation still lacks desired accuracy as local features, are still not matching the target shape very well e.g. the lesser and the greater trochanter. The following lesson explains how to capture more details and construct a better match.

## Incorporating a landmark- based non - linearity into the scaling law

The next level of detail can be achieved by utilizing a transform using the AnyFunTransform3DRBF class. This class is using radial basis functions (RBF) to represent the surface and is constructed by using source and target landmarks. Detailed behaviour of this transform is described in an appendix tutorial. However, the focus of this tutorial is to demonstrate available pipelines of transforms. For simplicity, we use a set of femur landmarks and RBF settings that were tested by the AnyBody™ team.

We start with the landmark based non-linear scaling by using the model of the previous steps. We will show how to build up scaling transforms by including them as pre-transforms and inherit achieved accuracy throughout different steps. A complete model can you find in lesson3b.Main.any . Let us add an RBF transform with the recommended settings into the previously created model.

First of all let us configure the visualization of the transformation. Now that we know how to compare source and scaled geometries as well as reverse registration, so we can switch off the registration step.

```
AnyFunTransform3DIdentity MyTransform3 =
{
PreTransforms = {&.MyTransform/*,&.MyTransform2*/};
};
```

This will return our scaled geometry back to the target bone location and we can compare the improvements we will create. Let us now define an RBF scaling law and another AnyDrawSurf object that will show the difference between the linear scaling law and the new created RBF scaling law. For a better contrast of the different surfaces, we will also add some colors to the drawing of the surfaces:

```
AnyDrawSurf SurfaceMorphedRBF =
{
FileName = "SourceFemur.stl";
AnyFunTransform3D &ref = ..MyRBFTransform;
RGB={1,0,0};
};
...
AnyFunTransform3DRBF MyRBFTransform =
{
PreTransforms = {&.MyTransform};
RBFDef =
{
Type = RBF_ThinPlate;
Param = 1;
};
Points0 = {
{ 0.0138, 0.0014, 0.0274},
{ 0.0791,-0.3971,-0.0524},
{ 0.0940,-0.3954,-0.0183},
{-0.0242,-0.0304,-0.0628},
{ 0.0381,-0.1886,-0.0388},
{ 0.0188,-0.3965,-0.0205},
{ 0.0369,-0.3937, 0.0267},
{-0.0127, 0.0039, 0.0290},
{ 0.0188, 0.0092,-0.0153},
{-0.0012, 0.0263, 0.0048},
{-0.0088,-0.0583, 0.0057},
{-0.0311,-0.0214,-0.0227},
{-0.0462,-0.0078,-0.0064},
{-0.0286,-0.0623,-0.0460},
{ 0.0010, 0.0013, 0.0069}
};
PointNames = {
"Medial_Head_Point",
"Anterior_LateralCondyle_Point",
"Anterior_MedialCondyle_Point",
"Anterior_GreaterTrochanter_Point",
"Anterior_Shaft_Point",
"Posterior_LateralCondyle_Point",
"Posterior_MedialCondyle_Point",
"Posterior_Head_Point",
"Anterior_Head_Point",
"Proximal_Head_Point",
"Medial_LesserTrochanter_Point",
"Distal_TrochantericFossa_Point",
"Proximal_Posterior_GreaterTrochanter_Point",
"Lateral_Lesser_Trochanter_Point",
"Femoral_COR"
};
Points1 = {
{ 0.2900, 0.4205, 0.0139},
{ 0.3220, 0.4332,-0.3786},
{ 0.2893, 0.4268,-0.3730},
{ 0.3599, 0.4429,-0.0050},
{ 0.3289, 0.4259,-0.1750},
{ 0.3062, 0.4872,-0.3703},
{ 0.2619, 0.4759,-0.3727},
{ 0.2900, 0.4405, 0.0139},
{ 0.3200, 0.4095, 0.0134},
{ 0.3100, 0.4295, 0.0314},
{ 0.3089, 0.4599,-0.0355},
{ 0.3349, 0.4579, 0.0050},
{ 0.3329, 0.4679, 0.0175},
{ 0.3519, 0.4599,-0.0355},
{ 0.3075, 0.4235, 0.0139}
};
BoundingBox =
{
Type = BB_Cartesian;
ScaleXYZ = {2, 2, 2};
DivisionFactorXYZ = 5*{1, 1, 1};
};
BoundingBoxOnOff = On;
};
```

**
**

The suggested transform morphs the source geometry in a thin-plate manner and tries to minimize the distance between the selected key points (landmarks). This can be useful if certain muscle attachment areas/points need to be scaled. Using this transform will make it possible to improve the model by morphing some local features that are important for the analysis. Please note that MyTransform object was included as a pre-transform as a rough scaling preceding the non-linear RBF function, and, therefore, the outcome of this transformation, our previous result, can be considered the first object in the transform pipeline. Target bone is color-coded with the green color, initial linear scaling is grey, RBF-scaled bone is red.

However, there is still some possibility to improve the fitting of the femur surfaces into each other and possibly improve the model. Looking at the Model View you can notice that the greater trochanter is elevated and makes a sharp corner this is due to the thin-plate nature of the transform and a low number of control points. The following section will describe how to utilize surface information for the construction of an improved scaling law.

## Incorporating a surface based non - linearity into the scaling law

In this section, we see how the scaling law can be
improved by utilizing two surfaces and requesting them to
fit each other, which will also deform all related muscle
and ligament attachment points accordingly. We use the
AnyFunTransform3DSTL class for this purpose. This class
constructs an RBF transform by using either corresponding
vertices on the STL surfaces or seeding a number of
vertices on one surface and finding a matching closest
point on the second. For constructing a transformation
using the vertices of STL surfaces, the surfaces have to be
topologically equivalent, i.e. the surfaces have the same
number of triangles and each neighbor and vertices are the
same on both surfaces. For the latter option, we require a
pretty good pre-registration transform, e.g. the RBF
transform that was described previously, in order for the
closest point finding to make sense. Due to the
implementation specifics most of the RBF recommendations
apply to this class as well. More details about how to
create this kind of transforms are described in the
appendix
tutorial**.**
However, for this example the recommended settings
mentioned before will be used again.

Let us repeat the step from the previous section by adding one more surface to the visualization and another scaling step. You can download the model with all modifications here :

```
AnyDrawSurf SurfaceMorphedSTL =
{
FileName = "SourceFemur.stl";
AnyFunTransform3D &ref = ..MySTLTransform;
RGB={0,0,1};
};
...
AnyFunTransform3DSTL MySTLTransform =
{
PreTransforms = {&.MyRBFTransform};
RBFDef.Type = RBF_ThinPlate;
FileName0 = "SourceFemur.stl";
//ScaleXYZ0 = {1, 1, 1};
FileName1 = "TargetFemur.stl";
ScaleXYZ1 = {1, 1, 1}*1;
NumPoints = 400;
BoundingBox.ScaleXYZ = {2, 2, 2};
BoundingBox.DivisionFactorXYZ = {1, 1, 1};
BoundingBoxOnOff = On;
};
```

Please note again the transform from the previous section of this tutorial was included as a pre-transform. Reloading the model, we can now see all steps of scaling in one place and try to switch them on and off. For example, let us try to hide affine and RBF scaled femurs to see the final results:

If we just look at the green target surface and the blue STL-transformed surface, we can see that the surfaces now match each other very well. That means that now the subject-specificity will be taken into account in the inverse dynamics simulation. The final model can be downloaded here .

Finally, the only thing left is to include this scaling law into an actual model. Lesson 4 describes how this can be done.