On March 19-20th 2019 AIR held its kick-out conference. All partners were present and presented the past years research and discoveries, as well as looking ahead by addressing future research challenges and applications. The conference was hosted by the Skövde University.
AIR research explained.
The general theme in AIR is on the interaction of humans and autonomous systems that move in shared physical spaces. The scenarios that instantiate this general theme are: autonomous systems in the home, autonomous systems in industrial environments, autonomous systems in the traffic and methodologies and framework. In that sense AIR is a one of a kind research project.
Discoveries and research of today
During day one, all research scenarios presented the last years research and discoveries. In the afternoon participants had the pleasure of listening to keynote Alessandra Sciutti from Istituto Italiano di Tecnologia (IIT). She spoke about about her research perspective on robots and humans from mutual understanding to interaction and trust.
Keynote day one, Alexandra Sciutti.
The research landscape and possibilities of tomorrow
Day two was spent at ASSAR – Industrial innovation arena. Participants where given a tour of the demonstrators displayed at ASSAR and demonstrators produced within the AIR project. These included industrial robots/rav that cooperate with man without security barriers, autonomous trucks, humanoids such as Pepper and FurHat, voice-controlled robots and VR technology.
Mingling images from day two.
Keynote day two, Bertil Thorvaldssson Senior Advisor at the University of Skövde and a Technical Fellow at ABB Robotics, give us a brief look into the past but for the main part of his talk The next wave of digital manufacturing he spoke about what lies ahead.
Keynote day two, Bertil Thorvaldsson.
Kick-off with open house
We ended the kick-out with an open house for the general public which came to be a very appreciated event. Both business associates and families came to learn more about the research conducted within AIR. In total this event had roughly 150 or so visitors. AIR started the open house by briefly explaining AIR and showing visitors below video produced:
Pepper and Rut exploring the technologies of CEJN production, Skövde.
On October 23rd researchers from the Istituto Italiano di Tecnologia (IIT) came to University of Skövde to discuss Human-robot interaction and to visit and be inspired at ASSAR – Industrial Innovation Arena.
IIT-RBCS has a long lasting activity on social robotics with special focus on investigating the role of social signs (such as gestures’ dynamic parameters and gaze direction). So on October 23rd the AIR consortia opened up the talk held by IIT to researchers.
Human-robot interaction The visiting researchers were Giulio Sandini, Alessandra Sciutti and Francesco Rea from the Robotics, Brain and Cognitive Sciences unit at IIT, in Genova, Italy. The researchers gave the attendants an insight to both the cognitive basis underlining social interaction and how social skills in robotics can be designed and implemented on the humanoid robot iCub to promote natural human-robot interaction.
Francesco Rea, Alessandra Sciutti and Giulio Sandini from the Robotics, Brain and Cognitive Sciences unit at IIT.
Tour of ASSAR After the talks, Jessica Lindblom and Erik Billing from the AIR team Skövde took Francesco, Alessandra and Giulio to ASSAR-industrial innovation arena. A meeting place where education, innovation and research supply industry with new solutions. The environment focuses on the development of production, production methods, and production technology as well as bear the stamp of diversity and equality—a meeting place where sustainable manufacturing will be created together. The name ASSAR is a historical tribute to the initiator of the Volvo companies, Assar Gabrielsson, but also a name which is looking to the future with the meaning “the one who gives answers.”
In all honesty, Anita Sant’Anna had her eyes set on being a medical doctor. However, after some time contemplating, she decided to study electrical engineering. Today she has managed to combine both her interests. Her research on health technology is about improving patients’ health, in close collaboration with companies, hospitals and patients.
Anita Sant’Anna, Assistant Professor at the School of Information Technology at Halmstad University.
Anita says, “I like to help people. When I find out what the needs and problems are in the health sector, it motivates me to continue researching. This means that I want to find the solutions to the problems.”
In her research, Anita Sant’Anna makes use of sensors, such as mobile phones and activity trackers, to collect or record patient health data. This can help doctors to get a clearer picture of how a patient feels or what the patient´s everyday life looks like. This can give forehand knowledge to help avoid an emergency situation developing. Furthermore, it gives the patient the opportunity to better healthcare through more careful monitoring and, therefore, better knowledge of his/her condition.
Wants to Improve the Health Care
The basis of Anita Sant’Anna’s research is about, together with several collaboration partners, improving the nursing and health of people. This is what drives her to continue to research, because it really does make a difference. However, the choice of becoming a researcher was not so clear-cut in the beginning for Anita Sant’Anna. She comes from a family where most work as engineers, and in fact, she chose to become an engineer herself, but with a focus on biomedicine. At the same time, she wanted to do something different. With an interest in both study and travel, she combined her studies with discovering new places. Finally, she ended up at Halmstad University, following a tip from a friend concerning a doctoral position that suited her perfectly: a combination of technology and health.
According to Anita Sant’Anna, one of the main challenges in health technology research is the huge amount of data collected.
During her doctoral studies, Anita Sant’Anna researched on how to monitor motion patterns by using sensors. She studied how patients walked with the help of sensors that were attached to the patients´ legs. By comparing how much deviation in the patient’s gait (style of walking) had to a healthy person´s way of walking, people working within the health care system could get a clearer picture of a patient’s illness or rehabilitation after, for example, an operation. Anita Sant’Anna has not only researched patients who have had hip replacements but she has also used sensors in research concerning people who suffer from Parkinson’s disease.
“Doctors do examine how well the patient walks, but often by using a rather “crude” basic scale. By studying how the patient walks with help of sensors, you get more knowledge that you may not be able to see by the naked eye alone. This can help create a clearer picture of the patient’s condition,” says Anita.
The more we work together with industry, companies and institutions, the better we will be able to solve the problems found in the health service. – Anita Sant’Anna
Health technique in focus
Anita Sant’Anna is still at Halmstad University. Her research is still about health technology. She is now researching how, with the help of technology, to help people suffering with osteoarthritis make healthier choices in their everyday lives. In her research, she uses the Fitbit activity tracker. “The challenge is how the patient will get suggestions for better choices in their everyday life through their activity tracker, in both a positive and motivating way. The goal is that it will happen at the right time, for example, proposing a ten-minute longer walk when the patient is already out walking,” she says.
One of the main challenges in health technology research is the huge amount of data that is collected or which is possible to collect. “It offers incredible opportunities. Nevertheless, at the same time, it is part of the challenge, because we need to find out what information we need, and then how to sort it out and be able to use it in the correct way,” say Anita.
Collaboration is important
Anita Sant’Anna wants to continue researching, in collaboration with both industry and other institutions. She believes it is important that research is not only undertaken in the lab but also with different collaborators; i.e. working together with patients in the hospital and also with contacts with industry, so that they can see the potential of new products. “The more we work together with industry, companies and institutions, the better we will be to solve healthcare problems.
September 21 every year is World Alzheimer’s Day. Alzheimer’s is the most common cause of dementia among older adults. Dementia is the loss of cognitive functioning – thinking, remembering, and reasoning – and behavioral abilities to such an extent that it interferes with a person’s daily life and activities. Halmstad University is part of a European initiative called Remind which develops technical solutions that can help patients with dementia and their relatives in everyday life.
Technical solutions can for example provide timely reminders for patients with dementia. Photo: iStock
Martin Cooney, a social robotics researcher at Halmstad University, is the local coordinator of Remind:
What is Remind? ”Remind is a four-year EU project involving staff exchanges between various partners from academia and industry in eight countries, within the area of reminding technologies for dementia and smart environments.”
What is the goal for Remind? ”The goal is to facilitate networking, training, and knowledge transfers through staff exchanges. For example, developers can learn about the user perspectives, and vice versa.”
How can this help a patient with Alzheimer’s? ”Participating staff are involved with various development and research projects, aimed to eventually contribute to a better quality of life for patients with Alzheimer’s. Currently we have, or are soon expecting, six people from Italy, Ireland, Norway, and Colombia at Halmstad University, and we are sending out three people to Spain soon.”
Martin Cooney, social robotics researcher at Halmstad University.
Examples of researchers within Remind and their projects
Usman Akhtar from Kyung Hee University, Korea, intends to at Halmstad University investigate a fog architecture for transfer learning related to activity recognition in smart homes. The goal is a better ability to monitor, share data, and provide timely reminders for patients with dementia.
Linda Ong from I+ srl, and Giovanni Nicoli from Associazione Novilunio, Italy, are interested in solutions involving robotic technology. For example, Linda has at Halmstad University started to incorporate sentiment analysis with a medical model of dementia into an interaction with a robot, which can provide appropriate and engaging reminders.
Dr. Jane Walsh, Director of the mHealth Research Group at National University of Ireland, Galway, and Dr. Riitta Hellman, Chairman Of The Board at Karde AS, Norway, work with users, in terms of behaviour change and tools for users with cognitive disabilities, respectively.
Dr. Miguel Angel Ortiz Barrios, coming to Halmstad University from Universidad de la Costa (CUC) in Colombia, works with data analytics (specifically statistical methods) and multi-criteria decision making.
Dr. Martin Cooney from Halmstad University has been working with thermal cameras as a way to infer medicine intake, and designs for socially assistive therapy robots, aimed to help meet a need for care which cannot currently be met due to lack of human healthcare workers.
Text and photo: LOUISE WANDEL, Halmstad University
When will we be able to teach robots to see like humans do? That means, to really interpret the surroundings and then make rational decisions. Today, research has not got that far. But Joseph Bigun, Professor of Signal Analysis, wants to be involved in the development of the first visually intelligent machine.
Perhaps vision is a human’s most important sense. We can manage without it, but it’s with our eyes most of us see and interpret the world.
“Our vision uses a great deal of our brain’s capacity. We need to process all the visual data and then put them into context. It really applies to anything we want to do. Our vision is so essential to our lives that we hardly think about what we would do without it.”
Josef Bigun is a Professor of Signal Analysis and Intelligent Systems. His research is about biometrics, that is technical means for computers to identify and collect information about us and our characteristics. It could be eye scanning, DNA, fingerprints or recognising a walking style. This involves teaching computers visual intelligence, and eventually teaching a robot to recognise images or patterns.
Everything can be used for good or bad purposes. But I think intelligent systems really will be able to help us in the future. I think it’s legitimate to be skeptical, that is why we should have better technology, make laws about using personal data and have good education about computers and artificial intelligence.
– Josef Bigun
Josef Bigun is a Professor of Signal Analysis and Intelligent Systems. His research is about biometrics, that is technical means for computers to identify and collect information about us and our characteristics.
“We already have computers with different types of intelligence, but it’s not sophisticated. Currently, there are no robots with a visual intelligence so well developed that they can draw conclusions like we humans do.”
Human vision is still largely superior to that of robots, but to succeed in making a robot copy the way we see the world is what fascinates and motivates Josef Bigun.
“It’s a big challenge to teach a robot to do what we’ve learnt since we were children,” he says.
Halmstad stands out
It was a bit of coincidence that Joseph Bigun ended up at Halmstad University. After graduating with a MSc in Engineering at Linköping University, he continued with a PhD in Computer Science at the same university. After ten years as a researcher at the Swiss Federal Institute of Technology, he returned to Sweden and received a position at Halmstad University, which has become one of the premier institutions in Sweden for research in visual intelligence and in particular, biometric identification.
The collaboration within the CAISR (Center for Applied Intelligent Systems Research) research group is important. The researchers discuss results and share ideas, and collaborate with many international colleagues. In 2016, CAISR organised the International Conference on Biometrics, with over 200 participants from around the world. It is one of the oldest and most respected conference regarding current research in biometric identification.
The research field in artificial intelligence is very active. It has also attracted much attention in media in recent years. While challenging, it’s primarily a lot of fun, Josef Bigun thinks.
“There’s a constant stream of new results published and things happening all the time. It has been like that for the past 40 years.”
Produce knowledge and solve problems
Josef Bigun’s goal as a researcher is clear. He would like to leave a better world, he says, and as a researcher he wants to contribute to continually improving both the environment and our lives.
“I don’t try to solve all the problems myself, but in my niche, I try to see how we can contribute. For example, by cameras that help to recycle and make environmental benefits, or to improve future transport with autonomous vehicles. I believe in the role of knowledge; new knowledge will help solve several problems.”
We are constantly building on our knowledge – it’s like an artist who keeps working on a piece of art.
– Josef Bigun
According th Josef Bigun, the best thing about working as a researcher is producing new knowledge.
He also hopes that his research will be useful for the robots of the future:
“Robots will need to help with different things in the future, where we won’t have enough human resources. Our society is still becoming more and more automated; what’s left to automate is often linked to visual intelligence.”
For example, robots can help in healthcare, but the technology can also be used with other types of services.
“As an example, cameras and sensors could become intelligent and beep if you haven’t turned off the stove. Or they could be used to check if a passenger has shown his ticket on a bus.”
A research field that sparks controversy
But it is not always easy to work with issues related to visual intelligence and intelligent computers. Sometimes the research is met with criticism and skepticism – or fear. Josef Bigun, on the other hand, argues that a future with visually intelligent robots is a bright future.
“Everything can be used for good or bad purposes. But I think intelligent systems really will be able to help us in the future. I think it’s legitimate to be skeptical, that is why we should have better technology, make laws about using personal data and have good education about computers and artificial intelligence. It’s much like cars: traffic rules, better vehicles and infrastructure and driving license training make traffic safer. We don’t ban the cars, the same goes for technology – by looking at the benefits and acting on several fronts we can minimise the risks.”
The best thing about working as a researcher is producing new knowledge, Josef Bigun thinks; and it’s something he can always do working in the research group CAISR.
“We are constantly developing new knowledge. It’s difficult to point to something that was particularly important in 2017. However, 2017 was better than all previous years because we are constantly building on our knowledge – it’s like an artist who keeps working on a piece of art”, he says.
People with a dementia diagnosis often struggle with daily routines, for example remembering to take their medication. In a European initiative called Remind, organisations from nine different countries collaborate to develop technical solutions that can help patients with dementia and their relatives in everyday life.
Remind is a European project where universities and companies collaborate to generate and share knowledge about health care solutions for patients with dementia.
Ubaid Ur Rehman is a PhD student from Kyung Hee University in South Korea, one of the partner universities in the Remind project. He visited Halmstad University as a guest researcher during three months in the beginning of 2018. Ubaid Ur Rehman has developed an application that, based on data from an intelligent home and a smartphone, can determine when it is a good time to remind a patient with dementia to take his or her medication.
– The main purpose of the project Remind is to generate and share knowledge between the involved organisations – which are both within the industry and the academy, says Ubaid Ur Rehman.
The Remind project gives us the opportunity to share ideas, information and results with others so that we efficiently can work towards the same goal.
– Anita Sant’Anna, Assistant Professor at the School of Information Technology at Halmstad University.
Towards the same goal
Remind was initiated by Anita Sant’Anna from Halmstad University and Chris Nugent from Ulster University in Northern Ireland. Chris Nugent was until recently a visiting professor at Halmstad University, focusing on development of mobile and pervasive computing solutions to support ambient assisted living.
Anita Sant’Anna, Assistant Professor at the School of Information Technology at Halmstad University, is one of the initiators of the Remind project. Photo: JOACHIM BRINK
– A growing elderly population requires new health technology solutions and smart home environments. The Remind project gives us the opportunity to share ideas, information and results with others so that we efficiently can work towards the same goal, says Anita Sant’Anna, Assistant Professor at the School of Information Technology at Halmstad University.
Martin Cooney, social robotics researcher at Halmstad University, is the local coordinator of Remind:
– Halmstad University is contributing to the project with competence in Artificial Intelligence. I think Ubaid’s activity recognition is a very good example of this. We have also, for example, made a system to detect recently performed activities – like taking medicine –using a thermal camera.
– Aside from the research, I think the project also offers nice opportunities for networking, by sending people to different countries, and connecting academia to industry, says Martin Cooney.
About Remind
Remind has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement. The project started in 2017 and will end in 2020. Remind webpage: https://www.remind-research.com
Partners:
Beneficiaries
Ulster University (UK)
University of Jaén (Spain)
National University of Ireland Galway (Ireland)
Luleå University of Technology (Sweden)
Halmstad University (Sweden)
University of Florence (Italy)
Associazione NOVILUNIO Onlus (Italy)
I+ srl (Italy)
KARDE AS (Norway)
Swedish Adrenaline (Sweden)
The Ageing social Lab Foundation (Spain)
Other organisations
University Medical Center Groningen (Netherlands)
Kyung Hee University (Korea)
Corporación Universidad de la Costa (Colombia)
Universidad Nacional Abierta y a Distancia (Colombia)
Knowledge exchange will be achieved through collaborative and hands-on activities where participants develop and/or implement something together within the following areas:
Signal and data analysis for healthcare
Context and behaviour modelling
User-centred reminding technologies
A final project demonstrator will be developed by the end of the project period, in June 2020.
Text: LOUISE WANDEL, Halmstad University
Film: IDA FRIDVALL and LOUISE WANDEL, Halmstad University
For humans, it is natural and relatively easy to interpret other people’s social signals and movement patterns, but when it comes to interacting with technical systems, like a robot or a driver-less car, there may be some challenges. A strong, distributed research environment has been granted funding from the KK-foundation to study human interaction with autonomous systems more closely.
In today’s society, there are already several so called autonomous systems, for example lawn mowers, household and industry robots, and cars that can park by themselves. But what exactly happens when autonomous systems try to interpret us humans, and how can we interpret their behaviour?
The KK-foundation has granted 27 million Swedish crowns over a period of four years to reserachers at Örebro University, the University of Skövde, the University of Halmstad and Viktoria Swedish ICT for a joint effort in a strong distributed research environment to study the interactions between humans and autonomous systems. Tom Ziemke, Professor of Cognitive Sciences at the University of Skövde is the lead researcher.
Using a map
– This is a multidisciplinary project, where reciprocation in the interaction is important. Interacting with a self-propelled lawn mower may seem as a simple interaction, but what about the interaction when it comes to a driver-less car taking a human to work? It is necessary that humans are able to interpret the artificial systems and the systems need to be able to interpret us for a successful interaction. This has all to do with trust and safety, like when driver-less cars actually operation in traffic, says Tom Ziemke, Professor at the University of Skövde.
– One example of what we will be working with in order to facilitate human-robot communication is outlines of maps. That is, how can a robot make use of human knowledge by using a map or a plan drawn on a piece of paper? says Martin Magnusson, researcher at Örebro University.
The researchers are also going to look at how robots can interpret human intentions by observing what they are doing – for example if they are sitting, walking, or standing in line – to enable a more intuitive interaction.
Instructions are not necessary
– That is, finding ways to bypass explicit instructions to the robot. Regarding information going the other direction, that is for the robot to behave in a manner that is easily understood by humans, we will study how a robot can communicate its plans and what it has been able to understand from its surroundings by projecting images on close objects, says Martin Magnusson.
– Another important step is to observe how robots can move in a way that is non-obstructive to humans, for example by teaching the robots what routes others use, to avoid the “traffic”.
The research environment will commence in the spring of 2015, and continue until 2019. The KK-foundation is a university funding entity with the purpose of strengthening Sweden’s competitiveness.
