Математический форум Math Help Planet
http://mathhelpplanet.com/

Функция mutiviewTriangulate в MATLAB, 3D Модель из 2Д точек
http://mathhelpplanet.com/viewtopic.php?f=23&t=80760
Страница 1 из 1

Автор:  K1b0rg [ 18 окт 2023, 22:55 ]
Заголовок сообщения:  Функция mutiviewTriangulate в MATLAB, 3D Модель из 2Д точек

Привет всем,
никак не разберусь что хочет функция mutiviewTriangulate .
То что я пытаюсь сделать:
Изображение

В общем есть изображения (12 штук) на каждом изображении есть 2д координаты точек каркаса (это сателлит), грубо говоря 8 2D (x,y) точек образуют параллелепипед плюс еще 3 точки (антенны). Изображения сделаны с разных позиций, есть расположение камеры для каждой картинки и intrinsic matrix камеры. Точки всегда в одном и томже порядке.


Вот параметры камеры:
% Create a camera intrinsics object
K = [
2988.5795163815555, 0, 960;
0, 2988.3401159176124, 600;
0, 0, 1
];


Далее идут 2Д координаты для 11 точек из 12 изображений, каждый массив (картинка) имеет 11 2Д точек:
% We have 12 images here. One array (line) is one image. 
% For example: [1025, 1015; 928, 937] are two 2D points from the first image.
landmarks_2d = {
[1025, 1015; 928, 937; 1252, 646; 939, 300; 630, 600; 957, 170; 510, 622; 817, 919; 1150, 611; 905, 330; 590, 654];
[1095, 948; 1206, 856; 742, 902; 935, 343; 1425, 360; 804, 221; 1528, 260; 1117, 678; 650, 667; 778, 269; 1244, 288];
[656, 177; 713, 229; 982, 327; 1199, 642; 940, 578; 1264, 730; 871, 645; 623, 419; 900, 460; 1049, 723; 778, 680];
[781, 643; 757, 717; 1021, 786; 1053, 1032; 803, 956; 1132, 1066; 762, 958; 832, 684; 1089, 749; 1108, 929; 866, 869];
[946, 510; 954, 611; 650, 623; 699, 1031; 1016, 989; 637, 1097; 1113, 1034; 1064, 648; 740, 660; 780, 974; 1013, 989];
[875, 614; 941, 568; 1023, 713; 1301, 675; 1228, 524; 1357, 709; 1253, 500; 980, 669; 1067, 800; 1272, 769; 1201, 634];
[862, 679; 942, 664; 904, 866; 1212, 895; 1225, 690; 1258, 926; 1274, 659; 987, 601; 948, 770; 1188, 787; 1202, 616];
[804, 228; 857, 279; 1157, 363; 1375, 688; 1078, 635; 1452, 775; 1002, 707; 775, 473; 1080, 500; 1231, 777; 924, 737];
[980, 262; 1105, 326; 768, 497; 1059, 882; 1415, 719; 997, 936; 1541, 757; 1087, 524; 734, 677; 956, 936; 1325, 816];
[1075, 347; 1086, 405; 872, 398; 861, 619; 1074, 626; 827, 650; 1124, 659; 1106, 448; 888, 443; 881, 611; 1097, 617];
[1063, 549; 1147, 474; 820, 325; 1016, 121; 1322, 252; 967, 85; 1420, 280; 1157, 544; 871, 420; 1010, 234; 1297, 361];
[1195, 502; 1128, 487; 1151, 274; 866, 210; 846, 417; 821, 158; 817, 476; 1053, 551; 1073, 333; 855, 283; 851, 502]
};


Далее идут позиции камеры относительно объекта для каждого изображения (всё консистентно landmarks_2d).
q это quaternion, r это translation vector..
q в таком формате: [w,x,y,z]
Поместил каждую pose в такой struct:

% The corresponding camera poses for the 12 images
poses = {
struct('q', [-0.358977, -0.163489, -0.01317, -0.918822], 'r', [-0.138635, 0.066346, 4.57807]);
struct('q', [0.27991, -0.907148, -0.0787, -0.304202], 'r', [-0.010698, -0.143755, 3.822349]);
struct('q', [-0.444023, -0.101919, -0.819316, 0.348107], 'r', [-0.206774, -0.047878, 4.985503]);
struct('q', [-0.262146, -0.020107, 0.91703, 0.299885], 'r', [0.026315, 0.499041, 7.490676]);
struct('q', [-0.946722, 0.081333, 0.309103, 0.039464], 'r', [-0.059765, 0.354944, 5.524356]);
struct('q', [-0.506775, -0.277448, -0.40971, 0.705932], 'r', [0.422576, 0.313776, 7.680882]);
struct('q', [-0.663318, 0.345193, 0.314251, 0.584891], 'r', [0.280749, 0.218569, 7.377274]);
struct('q', [0.387538, 0.090777, 0.84364, -0.36034], 'r', [0.066056, 0.034438, 4.883233]);
struct('q', [0.885113, 0.26256, 0.286374, -0.256178], 'r', [0.070995, 0.200342, 4.358347]);
struct('q', [0.989833, 0.081047, -0.115275, 0.019316], 'r', [0.115524, -0.246896, 9.844482]);
struct('q', [-0.348356, -0.898314, -0.258385, 0.070117], 'r', [0.22043, -0.417225, 6.27367]);
struct('q', [-0.590502, -0.367488, 0.163237, -0.699724], 'r', [0.010554, -0.450025, 7.690863])
};


Еще quaternion конвертирую в rotation matrix:
% Quaternion-to-rotation matrix conversion function
function R = quat2rotm(q)
% Adjusted for the quaternion format [w, x, y, z]
w = q(1); x = q(2); y = q(3); z = q(4);
R = [
1 - 2*(y^2 + z^2), 2*(x*y - z*w), 2*(x*z + y*w);
2*(x*y + z*w), 1 - 2*(x^2 + z^2), 2*(y*z - x*w);
2*(x*z - y*w), 2*(y*z + x*w), 1 - 2*(x^2 + y^2)
];
end


Читаю это документацию: https://de.mathworks.com/help/vision/ref/triangulatemultiview.html но что то не получается
По сути мне нужно вызвать функцию
Цитата:
worldPoints = triangulateMultiview(pointTracks, cameraPoses, intrinsics)

Не могу никак разобраться как подготовить данные для этой функции.

pointTracks делаю так:

% Number of images and landmarks
numImages = length(landmarks_2d);
numLandmarks = size(landmarks_2d{1}, 1);

% Array of pointTrack objects
tracks(numLandmarks) = pointTrack();

% View IDs for all 12 images
viewIDs = 1:numImages;

% Loop through each of the 11 landmarks
for i = 1:numLandmarks
% Extract the i-th landmark's coordinates across all 12 images
points = cell2mat(cellfun(@(c) c(i, :), landmarks_2d, 'UniformOutput', false));

% Create pointTrack object
tracks(i) = pointTrack(viewIDs, points);
end
% The 'tracks' array now contains 11 pointTrack objects, each with points across 12 views.


Теперь cameraPoses

numPoses = length(poses);
viewIDs = (1:numPoses).';
absolutePoses = cell(size(viewIDs));

for idx = 1:numPoses
% Convert quaternion to rotation matrix
q = quaternion(poses{idx}.q(1), poses{idx}.q(2), poses{idx}.q(3), poses{idx}.q(4));
R = rotmat(q, 'frame');

% Translation vector (make sure it's in row format)
t = poses{idx}.r;

% Convert to rigid3d object
absolutePoses{idx} = rigid3d(R, t);
end

% Create the table
cameraPoses = table(viewIDs, absolutePoses, 'VariableNames', {'ViewID', 'AbsolutePose'});


Для параметра intrinsics

focalLength = [2988.5795163815555, 2988.3401159176124];
principalPoint = [960, 600];
imageSize = [1200, 1920];

intrinsics = cameraIntrinsics(focalLength, principalPoint, imageSize);


Че то я тут не правильно сделал, мне кажется viewIDs = [1 2 3 4 5 6 7 8 9 10 11 12] должно быть.
И того, ошибка в том что я не правильно сделал cameraPoses таблицу. Начинаю исправлять в итоге еще больше ошибок. MATLAB использую первый раз.

Обновление, ошибок нету а модель не точная:
код без ошибок:
% Create a camera intrinsics object
K = [
2988.5795163815555, 0, 960;
0, 2988.3401159176124, 600;
0, 0, 1
];

focalLength = [2988.5795163815555, 2988.3401159176124];
principalPoint = [960, 600];
imageSize = [1200, 1920];

intrinsics = cameraIntrinsics(focalLength, principalPoint, imageSize);


% The corresponding camera poses for the 12 images
poses = {
struct('q', [-0.358977, -0.163489, -0.01317, -0.918822], 'r', [-0.138635, 0.066346, 4.57807]);
struct('q', [0.27991, -0.907148, -0.0787, -0.304202], 'r', [-0.010698, -0.143755, 3.822349]);
struct('q', [-0.444023, -0.101919, -0.819316, 0.348107], 'r', [-0.206774, -0.047878, 4.985503]);
struct('q', [-0.262146, -0.020107, 0.91703, 0.299885], 'r', [0.026315, 0.499041, 7.490676]);
struct('q', [-0.946722, 0.081333, 0.309103, 0.039464], 'r', [-0.059765, 0.354944, 5.524356]);
struct('q', [-0.506775, -0.277448, -0.40971, 0.705932], 'r', [0.422576, 0.313776, 7.680882]);
struct('q', [-0.663318, 0.345193, 0.314251, 0.584891], 'r', [0.280749, 0.218569, 7.377274]);
struct('q', [0.387538, 0.090777, 0.84364, -0.36034], 'r', [0.066056, 0.034438, 4.883233]);
struct('q', [0.885113, 0.26256, 0.286374, -0.256178], 'r', [0.070995, 0.200342, 4.358347]);
struct('q', [0.989833, 0.081047, -0.115275, 0.019316], 'r', [0.115524, -0.246896, 9.844482]);
struct('q', [-0.348356, -0.898314, -0.258385, 0.070117], 'r', [0.22043, -0.417225, 6.27367]);
struct('q', [-0.590502, -0.367488, 0.163237, -0.699724], 'r', [0.010554, -0.450025, 7.690863])
};


% We have 12 images here. One array is one image. For example: [1025, 1015; 928, 937] are two 2D
% points from first image.
landmarks_2d = {
[1025, 1015; 928, 937; 1252, 646; 939, 300; 630, 600; 957, 170; 510, 622; 817, 919; 1150, 611; 905, 330; 590, 654];
[1095, 948; 1206, 856; 742, 902; 935, 343; 1425, 360; 804, 221; 1528, 260; 1117, 678; 650, 667; 778, 269; 1244, 288];
[656, 177; 713, 229; 982, 327; 1199, 642; 940, 578; 1264, 730; 871, 645; 623, 419; 900, 460; 1049, 723; 778, 680];
[781, 643; 757, 717; 1021, 786; 1053, 1032; 803, 956; 1132, 1066; 762, 958; 832, 684; 1089, 749; 1108, 929; 866, 869];
[946, 510; 954, 611; 650, 623; 699, 1031; 1016, 989; 637, 1097; 1113, 1034; 1064, 648; 740, 660; 780, 974; 1013, 989];
[875, 614; 941, 568; 1023, 713; 1301, 675; 1228, 524; 1357, 709; 1253, 500; 980, 669; 1067, 800; 1272, 769; 1201, 634];
[862, 679; 942, 664; 904, 866; 1212, 895; 1225, 690; 1258, 926; 1274, 659; 987, 601; 948, 770; 1188, 787; 1202, 616];
[804, 228; 857, 279; 1157, 363; 1375, 688; 1078, 635; 1452, 775; 1002, 707; 775, 473; 1080, 500; 1231, 777; 924, 737];
[980, 262; 1105, 326; 768, 497; 1059, 882; 1415, 719; 997, 936; 1541, 757; 1087, 524; 734, 677; 956, 936; 1325, 816];
[1075, 347; 1086, 405; 872, 398; 861, 619; 1074, 626; 827, 650; 1124, 659; 1106, 448; 888, 443; 881, 611; 1097, 617];
[1063, 549; 1147, 474; 820, 325; 1016, 121; 1322, 252; 967, 85; 1420, 280; 1157, 544; 871, 420; 1010, 234; 1297, 361];
[1195, 502; 1128, 487; 1151, 274; 866, 210; 846, 417; 821, 158; 817, 476; 1053, 551; 1073, 333; 855, 283; 851, 502]
};

%-------------------->-==pointTracks==-<-----------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Number of images and landmarks
numImages = length(landmarks_2d);
numLandmarks = size(landmarks_2d{1}, 1);

% Array of pointTrack objects
tracks(numLandmarks) = pointTrack();

% View IDs for all 12 images
viewIDs = 1:numImages;

% Loop through each of the 11 landmarks
for i = 1:numLandmarks
% Extract the i-th landmark's coordinates across all 12 images
points = cell2mat(cellfun(@(c) c(i, :), landmarks_2d, 'UniformOutput', false));

% Create pointTrack object
tracks(i) = pointTrack(viewIDs, points);
end
% The 'tracks' array now contains 11 pointTrack objects, each with points across 12 views.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%-------------------->-==cameraPoses==-<-----------------------------------
% Assuming you have the poses array from earlier:
numPoses = numel(poses);
rigidTransforms = cell(1, numPoses);

for idx = 1:numPoses
normalized_quaternion = poses{idx}.q / norm(poses{idx}.q); % Normalize the quaternion
R = quat2rotm(normalized_quaternion);
t = poses{idx}.r;
rigidTransforms{idx} = rigid3d(R, t);
end

rigidTransformsArray = [rigidTransforms{:}];

% Now, create the cameraPoses table
cameraPoses = table(uint32(1:numPoses)', rigidTransformsArray', 'VariableNames', {'ViewId', 'AbsolutePose'});

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

worldPoints = triangulateMultiview(tracks, cameraPoses, intrinsics)

% Quaternion-to-rotation matrix conversion function
function R = quat2rotm(q)
% Adjusted for the quaternion format [w, x, y, z]
w = q(1); x = q(2); y = q(3); z = q(4);
R = [
1 - 2*(y^2 + z^2), 2*(x*y - z*w), 2*(x*z + y*w);
2*(x*y + z*w), 1 - 2*(x^2 + z^2), 2*(y*z - x*w);
2*(x*z - y*w), 2*(y*z + x*w), 1 - 2*(x^2 + y^2)
];
end

Автор:  K1b0rg [ 19 окт 2023, 21:20 ]
Заголовок сообщения:  Re: Функция mutiviewTriangulate в MATLAB, 3D Модель из 2Д точек

Решил, без той функции потому что взял функцию оптимизации из другой статьи. Но проблема там была точно такаяже.
Решение:

% Create a camera intrinsics object

format long;

K = [
2988.5795163815555, 0, 960;
0, 2988.3401159176124, 600;
0, 0, 1
];

% The ground truth poses for each image. Quaternion q is in this
% format: q = [x,y,z,w]
poses = struct('q', {}, 'r', {});
poses(1) = struct('q', [-0.358977, -0.163489, -0.01317, -0.918822], 'r', [-0.138635, 0.066346, 4.57807])
poses(2) = struct('q', [0.27991, -0.907148, -0.0787, -0.304202], 'r', [-0.010698, -0.143755, 3.822349]);
poses(3) = struct('q', [-0.444023, -0.101919, -0.819316, 0.348107], 'r', [-0.206774, -0.047878, 4.985503]);
poses(4) = struct('q', [-0.262146, -0.020107, 0.91703, 0.299885], 'r', [0.026315, 0.499041, 7.490676]);
poses(5) = struct('q', [-0.946722, 0.081333, 0.309103, 0.039464], 'r', [-0.059765, 0.354944, 5.524356]);
poses(6) = struct('q', [-0.506775, -0.277448, -0.40971, 0.705932], 'r', [0.422576, 0.313776, 7.680882]);
poses(7) = struct('q', [-0.663318, 0.345193, 0.314251, 0.584891], 'r', [0.280749, 0.218569, 7.377274]);
poses(8) = struct('q', [0.387538, 0.090777, 0.84364, -0.36034], 'r', [0.066056, 0.034438, 4.883233]);
poses(9) = struct('q', [0.885113, 0.26256, 0.286374, -0.256178], 'r', [0.070995, 0.200342, 4.358347]);
poses(10) = struct('q', [0.989833, 0.081047, -0.115275, 0.019316], 'r', [0.115524, -0.246896, 9.844482]);
poses(11) = struct('q', [-0.348356, -0.898314, -0.258385, 0.070117], 'r', [0.22043, -0.417225, 6.27367]);
poses(12) = struct('q', [-0.590502, -0.367488, 0.163237, -0.699724], 'r', [0.010554, -0.450025, 7.690863])


% The 2D coordinates. We have 12 images here. One array is one image. For example: [1025, 1015; 928, 937] are two 2D
% points from first image.
landmarks_2d = {
[1025, 1015; 928, 937; 1252, 646; 939, 300; 630, 600; 957, 170; 510, 622; 817, 919; 1150, 611; 905, 330; 590, 654];
[1095, 948; 1206, 856; 742, 902; 935, 343; 1425, 360; 804, 221; 1528, 260; 1117, 678; 650, 667; 778, 269; 1244, 288];
[656, 177; 713, 229; 982, 327; 1199, 642; 940, 578; 1264, 730; 871, 645; 623, 419; 900, 460; 1049, 723; 778, 680];
[781, 643; 757, 717; 1021, 786; 1053, 1032; 803, 956; 1132, 1066; 762, 958; 832, 684; 1089, 749; 1108, 929; 866, 869];
[946, 510; 954, 611; 650, 623; 699, 1031; 1016, 989; 637, 1097; 1113, 1034; 1064, 648; 740, 660; 780, 974; 1013, 989];
[875, 614; 941, 568; 1023, 713; 1301, 675; 1228, 524; 1357, 709; 1253, 500; 980, 669; 1067, 800; 1272, 769; 1201, 634];
[862, 679; 942, 664; 904, 866; 1212, 895; 1225, 690; 1258, 926; 1274, 659; 987, 601; 948, 770; 1188, 787; 1202, 616];
[804, 228; 857, 279; 1157, 363; 1375, 688; 1078, 635; 1452, 775; 1002, 707; 775, 473; 1080, 500; 1231, 777; 924, 737];
[980, 262; 1105, 326; 768, 497; 1059, 882; 1415, 719; 997, 936; 1541, 757; 1087, 524; 734, 677; 956, 936; 1325, 816];
[1075, 347; 1086, 405; 872, 398; 861, 619; 1074, 626; 827, 650; 1124, 659; 1106, 448; 888, 443; 881, 611; 1097, 617];
[1063, 549; 1147, 474; 820, 325; 1016, 121; 1322, 252; 967, 85; 1420, 280; 1157, 544; 871, 420; 1010, 234; 1297, 361];
[1195, 502; 1128, 487; 1151, 274; 866, 210; 846, 417; 821, 158; 817, 476; 1053, 551; 1073, 333; 855, 283; 851, 502]
};

% Compute projection matrices for each image
P = {};
for i = 1:length(poses)
R = quat2rot(poses(i).q);
t = poses(i).r';
Rt = [R, t];
P{i} = K * Rt;
end

% Triangulate points between consecutive image pairs
landmarks_3d = {};
for i = 1:length(landmarks_2d)-1
points1 = landmarks_2d{i};
points2 = landmarks_2d{i+1};
X = zeros(size(points1, 1), 3); % Corrected here

for j = 1:size(points1, 1)
X(j,:) = triangulate(points1(j, :), points2(j, :), P{i}, P{i+1});
end

landmarks_3d{i} = X;
end

% Accumulate the 3D coordinates for each landmark
accumulated_landmarks = zeros(size(landmarks_3d{1}, 1), 3);

for i = 1:length(landmarks_3d)
accumulated_landmarks = accumulated_landmarks + landmarks_3d{i};
end

% Compute the mean 3D coordinates for each landmark
mean_landmarks = accumulated_landmarks / length(landmarks_3d);

disp(mean_landmarks); % This will display an 11x3 array

% Now, landmarks_3d contains the 3D coordinates for each image pair

% Convert quaternion to rotation matrix
function R = quat2rot(q)
x = q(2); y = q(3); z = q(4); w = q(1);
R = [
1-2*y^2-2*z^2, 2*x*y-2*z*w, 2*x*z+2*y*w;
2*x*y+2*z*w, 1-2*x^2-2*z^2, 2*y*z-2*x*w;
2*x*z-2*y*w, 2*y*z+2*x*w, 1-2*x^2-2*y^2
];
end

Страница 1 из 1 Часовой пояс: UTC + 3 часа [ Летнее время ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/