{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden", "tags": [ "removecell" ] }, "outputs": [], "source": [ "import os\n", "os.chdir('/Users/mdl-admin/Desktop/mdl/')\n", "import imhr" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _roi: \n", " \n", ".. title:: Region of Interest\n" ] }, { "cell_type": "markdown", "metadata": { "toc-hr-collapsed": false }, "source": [ "### Region of Interest" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ":func:`imhr.eyetracking.ROI` can identify Region/Area of Interest using machine learning and image coding techniques." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. cssclass:: table-of-contents\n", "\n", ".. contents::\n", " :local:\n", " :depth: 3\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": false }, "source": [ "#### Introduction" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ "##### Example" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[45mnot multiprocessing\u001b[0m\n", "\u001b[45mstarting()\u001b[0m\n", "\u001b[45mfinished()\u001b[0m\n" ] } ], "source": [ "# import\n", "import imhr\n", "# parameters\n", "path = imhr.__path__[0]\n", "image_path = '%s/dist/roi/raw/2/'%(path)\n", "output_path = '%s/dist/roi/output/'%(path)\n", "metadata_source = '%s/dist/roi/raw/2/metadata.xlsx'%(path)\n", "position = 'topcenter'\n", "# initiate\n", "roi = imhr.eyetracking.ROI(isMultiprocessing=False, isDebug=True,\n", " # roi detection method and contour shape\n", " detection='manual', shape='straight',\n", " # path to read and export data\n", " image_path=image_path, output_path=output_path, metadata_source=metadata_source,\n", " # stimuli size and screen size\n", " screensize=[1920,1080], recenter=[(1920*.50),(1080*.50)], scale=.75,\n", " # applying new id, renaming variables and output\n", " newcolumn={'position': position}, uuid=['image','roi','position'], append_output_name=position, \n", " # image backend\n", " image_backend='PIL')\n", "# run\n", "df, error = roi.process()" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": true }, "source": [ "#### Parameters" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": false }, "source": [ "##### Required Parameters" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Here are all of the required parameters that are part of this module:" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. list-table::\n", " :class: kwargs\n", " :widths: 25 50\n", " :header-rows: 1\n", "\n", " * - Property\n", " - Description\n", " * - **isMultiprocessing** : :obj:`bool`\n", " - Should the rois be generated using multiprocessing. Default is **False**.\n", " * - **detection** : :obj:`str` {'manual', 'haarcascade'}\n", " - How should the regions of interest be detected. Either manually (**manual**), through the use of highlighting layers in photo-editing\n", " software, or automatically through feature detection using **haarcascade** classifers from opencv. Default **manual**.\n", " * - **image_path** : :obj:`str`\n", " - Image directory path.\n", " * - **output_path** : :class:`str`\n", " - Path to save data.\n", " * - **roi_format** : :obj:`str` {'raw', 'dataviewer', 'both'}\n", " - Format to export ROIs. Either to 'csv' (**raw**) or to Eyelink DataViewer 'ias' (**dataviewer**) or both (**both**).\n", " Default is **both**. Note: If **roi_format** = **dataviewer**, **shape** must be either be **circle**, **rotated**, or **straight**.\n", " * - **metadata_source** : :class:`str` or :obj:`None` {'path', 'embedded'}\n", " - Metadata source. If metadata is being read from a spreadsheet, **metadata_source** should be equal to path the to\n", " the metadata file, else if metadata is embed within the image as a layer name, **metadata_source** = **embedded**.\n", " Although Photoshop PSD don't directly provide support for metadata. However if each region of interest is stored\n", " as a seperate layer within a PSD, the layer name can be used to store metadata. To do this, the layer name has\n", " to be written as delimited text. Our code can read this data and extract relevant metadata. The delimiter can\n", " be either **;** **,** **|** **\\\\t** or **\\\\s** (Delimiter type must be identified when running this code using the\n", " **delimiter** parameter. The default is **;**.). Here's an example using **;** as a delimiter:\n", " \n", " .. rst-class:: code-param-whitespace\n", " \n", " >>> imagename = \"BM001\"; roiname = 1; feature = \"lefteye\"\n", " \n", " Note: whitespace should be avoided from from each layer name. Whitespaces may cause errors during parsing.\n", " * - **shape** : :obj:`str` {'polygon', 'hull', 'circle', 'rotated', 'straight'}\n", " - Shape of machine readable boundaries for region of interest. Default is **straight**. **polygon** creates a Contour\n", " Approximation and will most closely match the orginal shape of the roi. **hull** creates a Convex Hull, which\n", " is similar to but not as complex as a Contour Approximation and will include bulges for areas that are convex.\n", " **circle** creates a mininum enclosing circle. Finally, both **rotated** and **straight** create a Bounding Rectangle,\n", " with the only difference being compensation for the mininum enclosing area for the box when using **rotated**.\n", " * - **roicolumn** : :obj:`str`\n", " - The name of the label for the region of interest in your metadata. For example you may want to extract the column\n", " 'feature' from your metadata and use this as the label. Default is **roi**.\n", " * - **uuid** : :obj:`list` or :obj:`None`\n", " - Create a unique id by combining a list of existing variables in the metadata. This is recommended\n", " if **roi_format** == **dataviewer** because of the limited variables allowed for ias files. Default is **None**.\n", " * - **filetype**: :obj:`str` {'psd', 'tiff', 'dcm', 'png', 'bmp', 'jpg'}\n", " - The filetype extension of the image file. Case insensitive. Default is **psd**. If **psd**, **tiff** or **DICOM**\n", " the file can be read as multilayered\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": false }, "source": [ "##### Optional Parameters" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Here are optional parameters, not required to run the code, but allow output tweaking:" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Additional core parameters**" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "These properties control additional core parameters for the API:" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. list-table::\n", " :class: kwargs\n", " :widths: 25 50\n", " :header-rows: 1\n", "\n", " * - Property\n", " - Description\n", " * - **cores** : :obj:`bool`\n", " - (if **isMultiprocessing** == **True**) Number of cores to use. Default is total available cores - 1.\n", " * - **isLibrary** : :obj:`bool`\n", " - Check if required packages have been installed. Default is **False**.\n", " * - **isDebug** : :obj:`bool`\n", " - Allow flags to be visible. Default is **False**.\n", " * - **isDemo** : :obj:`bool`\n", " - Tests code with in-house images and metadata. Default is **False**.\n", " * - **save_data** : :obj:`bool`\n", " - Save coordinates. Default is **True**.\n", " * - **newcolumn** : :obj:`dict` {:obj:`str`, :obj:`str`} or :obj:`False`\n", " - Add additional column to metadata. This must be in the form of a dict in this form {key: value}. Default is **False**.\n", " * - **save_raw_image** : :obj:`bool`\n", " - Save images. Default is True.\n", " * - **append_output_name** : :obj:`bool` or :obj:`str`\n", " - Add appending name to all exported files (i.e. <'top_center'> IMG001_top_center.ias). Default is **False**.\n", " * - **save_contour_image** : :obj:`bool`\n", " - Save generated contours as images. Default is **True**.\n", " * - **scale** : :obj:`int`\n", " - If image is scaled during presentation, set scale. Default is **1**.\n", " * - **offset** : :obj:`list` [:obj:`int`]\n", " - Center point of image, relative to screensize. Default is **[960, 540]**.\n", " * - **screensize** : :obj:`list` [:obj:`int`]\n", " - Monitor size is being presented. Default is **[1920, 1080]**.\n" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Data processing parameters**" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "These properties control data is processed which include the type of haarcascade used, delimiters for metadata:" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. list-table::\n", " :class: kwargs\n", " :widths: 25 50\n", " :header-rows: 1\n", "\n", " * - Property\n", " - Description\n", " * - **delimiter** : :obj:`str` {';' , ',' , '|' , 'tab' , 'space'}\n", " - (if **source** == **psd**) How is metadata delimited. Default is **;**.\n", " * - **classifiers** : :obj:`default` or :obj:list of :obj:dict\n", " - (if **detection** == **haarcascade**) Trained classifiers to use. Default is {'eye_tree_eyeglasses', 'eye', 'frontalface_alt_tree', 'frontalface_alt', 'frontalface_alt2','frontalface_default', 'fullbody', 'lowerbody', 'profileface', 'smile', 'upperbody'}. Parameters are stored `here `__. If you want to use custom classifiers, you can pass a list of classifiers and their arguments using the following format:\n", "\n", " .. rst-class:: code-param-whitespace\n", "\n", " .. code-block:: python\n", "\n", " >>> [{'custom_cascade': { \n", " ... 'file': 'haarcascade_eye.xml', \n", " ... 'type': 'profileface',\n", " ... 'path': './haarcascade_eye.xml',\n", " ... 'minN': 5, \n", " ... 'minS':(100,100), \n", " ... 'sF': 1.01 }\n", " ... }]\n", "\n", " You can also pass custom arguments by calling them after initiation:\n", "\n", " .. rst-class:: code-param-whitespace\n", "\n", " .. code-block:: python\n", "\n", " >>> roi = imhr.eyetracking.ROI(detection='manual.....)\n", " >>> roi.default_classifiers['eye']['minNeighbors'] = 10\n", "\n" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Image export parameters**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Here are properties specific to how images are exported after processing. The code can either use :class:`matplotlib` \n", "or :class:`PIL` as a backend engine:" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. list-table::\n", " :class: kwargs\n", " :widths: 25 50\n", " :header-rows: 1\n", "\n", " * - Property\n", " - Description\n", " * - **image_backend** : :class:`str` {'matplotlib', 'PIL'}\n", " - Backend for exporting image. Either :class:`matplotlib` or :class:`PIL`. Default is :class:`matplotlib`.\n", " * - **RcParams** : :class:`bool`\n", " - A dictionary object including validation validating functions are defined and associated with rc parameters in class:`matplotlib.RcParams`. Default is **None**.\n", " * - **background_color** : :class:`list`\n", " - Set background color (RGB) for exporting images. Default is **[110, 110, 110]**.\n", " * - **dpi** : :class:`int` or :obj:`None`\n", " - (if **save_image** == **True**) Quality of exported images, refers to 'dots per inch'. Default is **300**.\n", " * - **remove_axis** : :class:`bool`\n", " - Remove axis from :obj:`matplotlib.pyplot`. Default is **False**.\n", " * - **tight_layout** : :class:`bool`\n", " - Remove whitespace from :obj:`matplotlib.pyplot`. Default is **False**.\n", " * - **set_size_inches** : :class:`bool`\n", " - Set size of :obj:`matplotlib.pyplot` according to screensize of ROI. Default is **False**.\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": true }, "source": [ "#### Creating ROI" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ "There are two methods of creating regions of interest to be processed by the api: using manually coded images or haar cascades." ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ "##### Manual coding" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: h4\n", "\n", "**Using multilayered images (psd, dcm)**" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ "The recommended method of manually identifying each region of interest is by using a image media file that is able to store multiple layers of content. The two filetypes supported here are Adobe Photoshop Document (**psd**) and Digital Imaging and Communications in Medicine (**dcm**) filetypes. Each ROI can be stored as a seperate layer, which can then processed for exporting." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", "**Figure:** Example **psd** in Adobe Photoshop.\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/example/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, ax = plt.subplots()\n", " # names\n", " file = 'photoshop_psd.png'\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " # labels\n", " ax.get_xaxis().set_ticks([])\n", " ax.get_yaxis().set_ticks([])\n", " # save\n", " plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " ## remove frame\n", " ax.axis('off')\n", " #plt.gca().axes.get_yaxis().set_visible(False)\n", " #plt.gca().axes.get_xaxis().set_visible(False)\n", " plt.tick_params(top='off', bottom='off', left='off', right='off', labelleft='off', labelbottom='off')\n", " plt.box(False)\n", " plt.show()\n" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", "**Figure:** Example **DICOM** in Adobe Photoshop.\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/example/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, ax = plt.subplots()\n", " # names\n", " file = 'photoshop_dicom.png'\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " # labels\n", " ax.get_xaxis().set_ticks([])\n", " ax.get_yaxis().set_ticks([])\n", " # save\n", " plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " ## remove frame\n", " ax.axis('off')\n", " #plt.gca().axes.get_yaxis().set_visible(False)\n", " #plt.gca().axes.get_xaxis().set_visible(False)\n", " plt.tick_params(top='off', bottom='off', left='off', right='off', labelleft='off', labelbottom='off')\n", " plt.box(False)\n", " plt.show()\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "##### Haar Cascades" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Another method for creating ROI's is by using automated feature extraction using [haar cascades](https://en.wikipedia.org/wiki/Haar-like_feature) feature classifiers. This is done by using machine-training to identify unique features (eyes, smile) from an image." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", "**Figure:** Example face, eye, and smile identfication using haar cascades.\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/example/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, ax = plt.subplots()\n", " # names\n", " file = 'haar_example.png'\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " # labels\n", " ax.get_xaxis().set_ticks([])\n", " ax.get_yaxis().set_ticks([])\n", " # save\n", " plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " ## remove frame\n", " ax.axis('off')\n", " #plt.gca().axes.get_yaxis().set_visible(False)\n", " #plt.gca().axes.get_xaxis().set_visible(False)\n", " plt.tick_params(top='off', bottom='off', left='off', right='off', labelleft='off', labelbottom='off')\n", " plt.box(False)\n", " plt.show()\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": false }, "source": [ "#### Output" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": false }, "source": [ ".. cssclass:: doctree\n", "\n", ".. raw::\n", "\n", "\n", " roi/ \n", " ├── output/ \n", " │ ├── bounds.ias \n", " │ ├── bounds.xlsx \n", " │ ├── data/ \n", " │ │ ├── 2550_bounds.ias \n", " │ │ └── 2550_bounds.xlsx \n", " │ └── img/\n", " │ ├── bounds/\n", " │ │ ├── 2550.png\n", " │ │ └── roi/\n", " │ │ ├── 2550.1.png\n", " │ │ └── 2550.2.png\n", " │ ├── preprocessed/\n", " │ │ └── 2550.png\n", " │ └── raw/\n", " │ └── 2550.png\n", " └── raw/\n", " ├── 1/\n", " │ ├── AM_201.psd\n", " │ ├── AM_201.xcf\n", " │ └── metadata.xlsx\n", " └── 2/\n", " ├── 2550.psd\n", " ├── 2550.xcf\n", " └── metadata.xlsx\n" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ "After running the API, the following data will be outputted:\n", "\n", "* Rerendered images (**img**)\n", " * Dataviewer Region of Interest (**ias**): './output/*.ias', './output/data/*.ias' \n", "* Data\n", " * Dataviewer Region of Interest (**ias**): './output/*.ias', './output/data/*.ias' \n", " * Excel Region of Interest (**xlsx**): './output/*.xlsx', './output/data/*.xlsx' \n", "\n", "In addition, for each image, their corresponding ROIs will be drawn to a png file (./img/bounds). Individual ROIs are also drawn ('./img/bounds/roi'). " ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown", "toc-hr-collapsed": true }, "source": [ "##### Bounds" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ "Bounds can be created for each region of interest with different levels of granularity. The list below provides an example of each, in order of granularity." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** Original images before contour extraction." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/example/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'raw'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Polygon**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** Contour Approximation (**polygon**) that will most closely match the orginal shape of the ROI." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/output/img/bounds/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'polygon'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ ".. rst-class:: h4\n", "\n", "**Hull**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** Generated Convex Hull (**hull**), which is similar to but not as complex as a Contour Approximation and will include bulges for areas that are convex." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/output/img/bounds/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'hull'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Circle**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** **Circle** creates a mininum enclosing circle, with the center equal to that of the ROI." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/output/img/bounds/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'circle'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: h4\n", "\n", "**Rectangle (rotated)**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** Bounding Rectangle (**rotated**), with the position matching that of the general shape of the ROI." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/output/img/bounds/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'rotated'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()\n" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ ".. rst-class:: h4\n", "\n", "**Rectangle (straight)**" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. rst-class:: caption\n", "\n", "**Figure:** Bounding Rectangle (**straight**) parallel to the shape of the background ima" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ ".. cssclass: figure\n", "\n", ".. plot::\n", "\n", " import sys\n", " # add source\n", " sys.path.insert(0, os.path.abspath('./sphinxext'))\n", " sys.path.append('/Users/mdl-admin/Desktop/mdl')\n", " # import package\n", " import imhr\n", "\n", " # create image\n", " import matplotlib.pyplot as plt\n", " import matplotlib.image as image\n", " from pathlib import Path\n", " # path\n", " path = '%s/dist/roi/output/img/bounds/'%(Path(imhr.__file__).parent)\n", " # draw plot\n", " #plt.figure(figsize=(20,6), dpi=400, facecolor='#ffffff')\n", " fig, (axes) = plt.subplots(1, 4, sharey=True)\n", " # names\n", " shape = 'straight'\n", " filenames = ['2550_%s.png'%(shape),'2691_%s.png'%(shape),'4640_%s.png'%(shape),'9421_%s.png'%(shape)]\n", " # draw and save\n", " for idx, itm in enumerate(zip(axes, filenames)):\n", " ax, file, = itm\n", " ## load roi\n", " im = image.imread('%s/%s'%(path, file))\n", " ax.imshow(im)\n", " ax.grid(True)\n", " ax.set_facecolor('#f9f9f9')\n", " # labels\n", " if idx == 0: ax.set_ylabel('Screen Y (pixels)', fontsize=8)\n", " ax.set_xlabel('Screen X (pixels)', fontsize=8)\n", " ax.tick_params(labelsize=6, width=1, length=4)\n", " # save\n", " #plt.tight_layout()\n", " plt.subplots_adjust(wspace=0.1)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": true }, "source": [ "##### Data" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/restructuredtext", "toc-hr-collapsed": false }, "source": [ "Two types of data are exported, contour shape and boundaries. Boundaries are saved in SR Research DataViewer (**ias**), and Microsoft Office Open XML Workbook (**xlsx**) formats, while contour shapes are saved in Hierarchical Data Format (**HDF5**; **h5**). Exported data can read either by SR Research DataViewer (**ias**) or any other data processing tool (**xlsx**, **h5**). \n", "\n", "
\n", "

**Note**

\n", "

\n", " Section 5.10.1 of Eyelink DataViewer Users Manual (3.2.1)\n", "

has more information about the ias format and how DataViewer can interpret it.
\n", "

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both **xlsx** and **HDF5** provide all metadata associated with each image, while **ias** files matches the format required by DataViewer. Of the three filetypes, HDF5 files are the most detailed providing both metadata for each ROI as well as exact pixel coordinates associated with each ROI. \n", "\n", "For more information, click [bounds.ias](./bounds.ias), [bounds.xlsx](./bounds.xlsx), [contours.h5](./contours.h5) to see an example of each filetype." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "nbsphinx": "hidden", "tags": [ "removecell" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[45mnot multiprocessing\u001b[0m\n", "\u001b[45mstarting()\u001b[0m\n", "\u001b[45mfinished()\u001b[0m\n" ] } ], "source": [ "#import\n", "import imhr\n", "#initiate\n", "roi = imhr.eyetracking.ROI(isDemo=True)\n", "#run\n", "df, error = roi.process();" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. cssclass: figure\n" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rst-class:: caption\n", "\n", " **Figure:** Sample bounds output saved to xlsx.\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
x0y0x1y1imageroiidshape_dracefeature
0937554792422AM_201roi11RECTANGLEasianlefteye
01118552973420AM_201roi22RECTANGLEasianrighteye
01016650892565AM_201roi33RECTANGLEasiannose
08807561039670AM_201roi44RECTANGLEasianmouth
01114862818762AM_201roi55RECTANGLEasianchin
01109407801307AM_201roi66RECTANGLEasianforehead
0783665704532AM_201roi77RECTANGLEasianleftear
012076491144494AM_201roi88RECTANGLEasianrightear
0938524785409AM_203roi11RECTANGLEasianlefteye
01125525976409AM_203roi22RECTANGLEasianrighteye
01023632899552AM_203roi33RECTANGLEasiannose
01043735876661AM_203roi44RECTANGLEasianmouth
01040837862759AM_203roi55RECTANGLEasianchin
01137382778259AM_203roi66RECTANGLEasianforehead
0766637693474AM_203roi77RECTANGLEasianleftear
012136341147462AM_203roi88RECTANGLEasianrightear
0928538778425AM_204roi11RECTANGLEasianlefteye
01124535975427AM_204roi22RECTANGLEasianrighteye
01022633886567AM_204roi33RECTANGLEasiannose
08627351050649AM_204roi44RECTANGLEasianmouth
08748311038755AM_204roi55RECTANGLEasianchin
01137410776262AM_204roi66RECTANGLEasianforehead
0766658700483AM_204roi77RECTANGLEasianleftear
012176441148483AM_204roi88RECTANGLEasianrightear
0944531792406AM_223roi11RECTANGLEasianlefteye
01132524983405AM_223roi22RECTANGLEasianrighteye
01023641906562AM_223roi33RECTANGLEasiannose
01038741896670AM_223roi44RECTANGLEasianmouth
01053847875776AM_223roi55RECTANGLEasianchin
01150386778275AM_223roi66RECTANGLEasianforehead
0775641695488AM_223roi77RECTANGLEasianleftear
012266381145483AM_223roi88RECTANGLEasianrightear
\n", "
" ], "text/plain": [ " x0 y0 x1 y1 image roi id shape_d race feature\n", "0 937 554 792 422 AM_201 roi1 1 RECTANGLE asian lefteye\n", "0 1118 552 973 420 AM_201 roi2 2 RECTANGLE asian righteye\n", "0 1016 650 892 565 AM_201 roi3 3 RECTANGLE asian nose\n", "0 880 756 1039 670 AM_201 roi4 4 RECTANGLE asian mouth\n", "0 1114 862 818 762 AM_201 roi5 5 RECTANGLE asian chin\n", "0 1109 407 801 307 AM_201 roi6 6 RECTANGLE asian forehead\n", "0 783 665 704 532 AM_201 roi7 7 RECTANGLE asian leftear\n", "0 1207 649 1144 494 AM_201 roi8 8 RECTANGLE asian rightear\n", "0 938 524 785 409 AM_203 roi1 1 RECTANGLE asian lefteye\n", "0 1125 525 976 409 AM_203 roi2 2 RECTANGLE asian righteye\n", "0 1023 632 899 552 AM_203 roi3 3 RECTANGLE asian nose\n", "0 1043 735 876 661 AM_203 roi4 4 RECTANGLE asian mouth\n", "0 1040 837 862 759 AM_203 roi5 5 RECTANGLE asian chin\n", "0 1137 382 778 259 AM_203 roi6 6 RECTANGLE asian forehead\n", "0 766 637 693 474 AM_203 roi7 7 RECTANGLE asian leftear\n", "0 1213 634 1147 462 AM_203 roi8 8 RECTANGLE asian rightear\n", "0 928 538 778 425 AM_204 roi1 1 RECTANGLE asian lefteye\n", "0 1124 535 975 427 AM_204 roi2 2 RECTANGLE asian righteye\n", "0 1022 633 886 567 AM_204 roi3 3 RECTANGLE asian nose\n", "0 862 735 1050 649 AM_204 roi4 4 RECTANGLE asian mouth\n", "0 874 831 1038 755 AM_204 roi5 5 RECTANGLE asian chin\n", "0 1137 410 776 262 AM_204 roi6 6 RECTANGLE asian forehead\n", "0 766 658 700 483 AM_204 roi7 7 RECTANGLE asian leftear\n", "0 1217 644 1148 483 AM_204 roi8 8 RECTANGLE asian rightear\n", "0 944 531 792 406 AM_223 roi1 1 RECTANGLE asian lefteye\n", "0 1132 524 983 405 AM_223 roi2 2 RECTANGLE asian righteye\n", "0 1023 641 906 562 AM_223 roi3 3 RECTANGLE asian nose\n", "0 1038 741 896 670 AM_223 roi4 4 RECTANGLE asian mouth\n", "0 1053 847 875 776 AM_223 roi5 5 RECTANGLE asian chin\n", "0 1150 386 778 275 AM_223 roi6 6 RECTANGLE asian forehead\n", "0 775 641 695 488 AM_223 roi7 7 RECTANGLE asian leftear\n", "0 1226 638 1145 483 AM_223 roi8 8 RECTANGLE asian rightear" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df" ] } ], "metadata": { "celltoolbar": "Edit Metadata", "file_extension": ".py", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" }, "mimetype": "text/x-python", "name": "python", "npconvert_exporter": "python", "pygments_lexer": "ipython3", "toc-showcode": false, "toc-showtags": false, "version": 3 }, "nbformat": 4, "nbformat_minor": 2 }