PickGPT takes advantage of similar technologies used in Open AI’s ChatGPT, including large language models.
By Robotics 24/7 Staff July 5, 2023
Recently, Sereact announced the release of PickGPT – a robotics transformer that combines large language models, or LLMs, with computer vision. The company said PickGPT opens the possibility of instructing robots using natural language.
The Robotics Transformer can recognize and identify objects and products in real time based on their visual characteristics. PickGPT automatically analyzes product images and matches them with items in any real-world scenario.
In addition, PickGPT understands and interprets product titles, descriptions, and other textual information to automatically identify and assign corresponding products to regions in an image. Especially for quality control, this capability offers enormous potential. This enables robots to detect incorrect or defective products and blacklisted items.
With this technology, Stuttgart-based Sereact claims it is offering one of the easiest ways to interact with robots, making them accessible to anyone. Especially in the logistics industry, PickGPT can solve critical problems such as returns processing, the company added.
LLMs have been in focus since the introduction of ChatGPT and offer great potential when combined with other technologies, the company explained. Sereact noted it has recognized this potential: PickGPT combines the power of LLMs, similar to the models used in ChatGPT, with Sereact’s patented computer vision models.
This enables robots to process natural language and visual information and correlate multimodal data. They can perceive their environment with unprecedented intelligence and accuracy and understand and execute instructions in natural language.
Natural cooperation between humans and robots
Employees can now use voice-based instructions or intuitive user interfaces instead of requiring complex programming skills to control robots.
On the one hand, this reduces the time needed to set up and customize robots and allows employees to focus on other essential tasks. On the other hand, the intuitive interface makes advanced robotics technology accessible to people without prior knowledge or training, which expands the range of people operating robots in the warehouse.
“PickGPT enables straightforward communication between humans and robots via voice commands and intuitive interfaces. This simplifies the link with other warehouse systems and enables employees to instruct the robot in a simple way – for example, to define pick points or to blacklist certain items,” said Ralf Gulde, co-Founder and CEO of Sereact.
Flexibility and quality control
PickGPT enables robots to respond quickly and easily to new tasks or changing environmental conditions. This is particularly important in dynamic and rapidly changing work environments, which are especially common in the logistics industry.
The ability to instruct robots intuitively enables companies to design their work processes flexibly and respond quickly to changes.
The processing of returns presents many e-commerce companies with major challenges. By automating this process, PickGPT helps to save time and resources.