**6. Organ tracking**

An ideal AR system would allow the organ to be tracked in real time as it is affected by respiration, tissue deformation and other complications like bleeding. The renal system is particularly vulnerable to these dynamic changes compared to other organs systems like the brain or bones where rigid image registration systems are the norm and do not require as much tracking. The registered image projected into the operative view should be locked and dynamically move with the organ and the laparoscopic camera.

There are many types of tracking studied in AR surgery. These have mainly included optical tracking where optical markers on the organ allow a measured position of the laparoscopic camera relative to the organ and move with the organ [20]. Infra-red tracking is another method which involves the use of infrared-emitting diode markers. The main issue with optical tracking is instruments obscuring the direct view of field (required for tracking) and a limited depth perception. Infra-red tracking has the issue of selecting the correct anatomical landmarks as markers—mismatches can occur due to deformations, compression and intraoperative haemorrhaging. Many studies have failed to achieve accurate registration of dynamic intra-abdominal organs with infra-red [21].

Electromagnetic tracking is another way of doing this—this has been explored by use of the wireless trackers an ex-vivo bovine partial nephrectomy model. This involved surrounding the tumour within the kidney with magnetic transponders which relayed back to the surgeon and in conjunction with optical camera tracking, a partial nephrectomy was performed [22]. There are limitations to this tracking method as magnetic fields can have interference from laparoscopic instruments and operating tables. The method also requires placement of the magnetic transponders into the target organ and it is currently hard to achieve an alignment error of less than 5 mm—a tumour margin error too great for accurate partial nephrectomies [23].

An alternative has been explored by Yip et al. where 3D stereoscopic image registration has been combined this with tracking algorithms—producing only 1.3–3.3 mm degree of error [24]. More recently, Edgcumbe et al. have developed a tracking device called the Dynamic Augmented Reality Tracker (DART). This is a 3D-printed stainless-steel tracker that can be anchored to a fixed position on the kidney relative to the tumour. This, with the help of an ultrasound transducer, can then be used to track the location of surgical instruments relative to the tumour in real time. This system has been named ARUNS (Augmented Reality Ultrasound Navigation System) and was used in the robotic-assisted excision of a phantom kidney tumour [25].

Vávra et al., in their recent review, comment that it may be possible to track organ movement without physical markers in the future. Some of these methods explored include algorithms to predict real-time movement of organs, physics-based deformation models, natural points of reference as tracking points and the use of red-green-blue cameras to perform image registration without markers. They also comment that whilst the average marker associated registration takes 8 min, a recent marker-less system only took about 5 min [20].
