Posts in KPI

Mitigate Your FMEA R&D Risks By Building Smart KPI Views To Measure R&D Performance

January 18th, 2016 Posted by KPI 0 thoughts on “Mitigate Your FMEA R&D Risks By Building Smart KPI Views To Measure R&D Performance”

Post Overview

Whether you are a medical device vendor or non-regulated SVP, most of your resources should be spent on your research and development department. For this reason your RISK Management assessments process, based on FMEA, should consider development RISKs. You need to create Key Performance Indicators (KPI), which are based on your organizational context. You will need to define methodology which measures the value created by R&D activities, which will essentially allow you  to:

  • Capture not only merely economic value, but intangible assets;
  • Measure the efficiency and effectiveness of ongoing projects in the R&D portfolio, focusing on the most relevant;
  • Assure transparent and traceable estimates, based on actual data;
  • Assess regularly the coherence between feature development and defect sensitivity;
  • Support the evaluation of different kinds of projects and results: Incremental vs. breakthrough, core business technologies vs. emerging ones;
  • Evaluate results, at least once a year, perform post postmortems;
  • Support the planning and leadership role of Corporate in the R&D activities.

So as the development team leader or R&D manager, you are responsible for examining your defect history log, and identifying those areas in the product that are the most difficult to fix. Such a survey is beneficial in predicting or mitigating the recurrence of such events.

2016-01-16_21-40-25

 

How Do I Create Historical Measurement On Defects In QPack?

In QPack View Designer, there are many options that enable you to check the current state of work items. However, should you need to create a view that is multidimensional, then it is beneficial to apply the  User Defined Criteria, which will allow you to review historical data in Orcanos ALM. So essentially, your initial investigation will uncover defects within your population, while your  second inquiry will be on the historical values of  said population. There you can place an additional WHERE clause

2016-01-16_22-01-05

For example:

id in (select obj_id

from sys_history

where tablechanged=’DEFECTS’ and fieldname=’status’

group by obj_id, cast(NewValue as nvarchar(max))

having count(*)>1 and cast(NewValue as nvarchar(max))=‘REOPEN’)

The SQL statement checks changes in the defect history and returns those who were REOPENED more thanonce. You may want to do the same for those that were REOPENED more than twice. Here are the results you can expect

More Advanced Investigation In Orcanos ALM

Now that we have conducted that analysis we are able to do more in-depth research and determine what aspects of our products suffer the most from REOPEN events. Additionally, we can use the double reopen research to refine those areas and find the most critical areas of functions in our project. Should you create a second chart view of that data where you have placed your feature list that those defects are associated with, you may get interesting results.

2016-01-16_22-09-26

From the first REOPEN, it is clear which areas, in general, are more difficult to resolve doing it First Time Right. From the second report, you may find which areas require extra effort to solve using those second REOPEN, which highlight the low efficiency areas in problem resolution.  If some methodology of Root Cause Analysis is being used in your R&D activity, you will have not just a view of your problem, but a view of your solution and mitigation of such events.

Tip Of The Week – How to Build Your Design Inputs/Outputs in QPack According to ISO 13485

November 7th, 2015 Posted by ALM 2.0, KPI, Tip Of The Week 0 thoughts on “Tip Of The Week – How to Build Your Design Inputs/Outputs in QPack According to ISO 13485”

October 25th 2015 335d499  Rami Azulay in Tip of the Week, QPack


Start Your Free Trial Now


 

Post Overview

If you are a medical device vendor, then you are probably aware of how the ISO 13485 methods of design and development. In the design phase, we draw on  all our market and product requirements, which are  used in designing. This data also informs  the project plan. Later comes the specifications, which are also determined by the  design inputs, which could be Software/Hardware/Mechanical requirements and their coupling detail design.

How Do I Create Design Inputs In QPack?

In QPack there are 36 work items that have different processes and procedures. Two of them belong to the design input phase. One is the Business Requirement work item. This work item is used to describe the business goals of the product. It is not yet a mandatory document for all medical device vendors, but if your medical device is software (Ex. Mobile Phone App.) only then you must start with that document. Yet best practice demonstrates that most successful projects start with thinking about market needs, and then goes downstream to defining the product.

2015-10-31_14-29-56

 

 

This is  a recommended template, which may be used to increase user productivity and focus on market goals, rather than features:

 

Field Description
MR Identifier QPack ID
MR Name Item Name
Directive

<Provide the instruction that guides and directs functionality being sought by the user.  Each directive addresses a facet of the market problem.  The directive format is:

“User [persona] <shall/should[(high/medium/low)]> be able to <functionality>.”>

<Comment: The special conventions used in the directive are:

  • Square brackets “[ ]” denote optional arguments
  • Pointed brackets “<>” denote mandatory arguments
  • Slash symbol “/” denote the “OR” logical operator.>
Priority QPack Item Priority
  • Rationales
<Provide a list of all possible rationales; the reasons that support the introduction of this market requirement.>
  • Sources
<Each rationale must be supported by a source.  Sources are a list of references and information origins that validate the market requirement.>
Constraints <Provide a list of all possible constraints; the limitations imposed on the solution relevant to this particular market requirement.  Each constraint should be supported by its own rationale and source.>
  • Rationales
<Provide a list of all possible rationales; the reasons that support the introduction of this constraint.>
  • Sources
<Each rationale must be supported by a source.  Sources are a list of references and information origins that validate the market requirement.>
Persona <List names of all personas applicable to this market requirement.>
Use Case <Provide a use case statement or use case identifiers applicable to this market requirement.  Entry of use case information is applicable only if a product or product concept actually exists.>
Buying Criterion <Indicate via this Boolean indicator (Yes/No/NA) whether this market requirement will foster a product feature used as a buying criterion.>
Differentiator <Indicate via this Boolean indicator (Yes/No/NA) whether this market requirement will foster a product feature that is a key differentiator, relative to competing products.>

 

In addition, QPack Product Requirements  can also be applied as a design input requirement. According to ISO 13485 section 7.3.2, design inputs relating to product requirements shall be determined and records maintained (see 4.2.4). These inputs include the following information

  1. a) Functional, performance and safety requirements, according to the intended use,
  2. b) Applicable statutory and regulatory requirements,
  3. c) Where applicable, information derived from previous similar designs,
  4. d) Other requirements essential for design and development, and
  5. e) Output(s) of risk management (see 7.1).

These inputs are reviewed for adequacy over PDR and CDR, and approved. Requirements are complete, unambiguous and not in conflict with each other.  

How Do I Create Design Output In QPack?

According to ISO 13485 Sec. 7.3.3, the outputs of design and development are inserted on a form, which also  enables verification ,and is approved prior to release. Design and development outputs:

  1. Meet the input requirements for design and development                                                                                                                                                               2015-10-31_21-56-25
  2. Provide appropriate information for purchasing, production and for service provision
  3. Contain or reference product acceptance criteria                                                                                                                                                                              2015-10-31_22-02-12
  4. Specify the characteristics of the product that are essential for its safe and proper use.

 

QPack FMEA Section

Risk Product Characteristic

Records of the design and development outputs are maintained (see ISO 13485 Sec. 4.2.4).

NOTE: Records of design and development outputs may include software/hardware specifications, manufacturing procedures, engineering drawings, and engineering or research logbooks.

In QPack you have several sets of work items that can be used to describe the design outputs. In the case of software, the system include work items such as Software Requirements/Software Detail Design/Firmware Requirement/Firmware Detail Design. In relation to hardware design outputs, the system includes Hardware Requirement/Hardware Detail Design, and in the case of mechanical design outputs, there are Mechanical Requirements/Mechanical Details Design (Drawings). All 8 types of work items complete any possible medical device design output.

 

Tip Of The Week – Use QPack ALERTS To Keeps People On The Same Page

October 24th, 2015 Posted by ALM 2.0, KPI, Tip Of The Week 0 thoughts on “Tip Of The Week – Use QPack ALERTS To Keeps People On The Same Page”

October 25th 2015 2015-10-24_07-14-47 Amita Gupta in Tip of the Week, QPack


Start Your Free Trial Now


 

 

Use QPack ALERTS To Keeps People On The Same Page

Alerts are very powerful tools useful in keeping track of updates or anything new that happens during the process of Application Life-cycle Development.  Alerts can be configured to signal any type of information in well organized manner. It can also be configured whether you want them to be delivered or not and when. This is done simply by creating views, and by using those views, one can create his own alert to send those views results as email notification to selected  a preset audience. Alerts minimize  the need to gather  updates from each individual in the organisation. For example, alerts are beneficial to a  manager who may want a daily update on how much testing is done, and which requirements have been covered by teams. So instead each team member emailing  reports on completed tasks,, the manager can simply create an alert that collects  the information about the work being done by the team members and schedule that to come in before every morning meeting. So time spent on meeting is less and important issues are raised first.

How Alerts Can Serve Quality Manager?

As a quality manager, there are many events within an organization that requires strong documentation and actions. One of the best uses of QPack Alerts, is creating  what is called Pre Audit alerts. These alerts serve as watchdog agents within the quality process. inconsistencies such as the following, can be found:

  • Broken Traceability
  • RISK without Mitigation
  • Failure in the V&V process
  • Complaints out of SLA
  • CAPA with overdue actions items etc.

How to Create Alerts?  

The Admin user can create alerts by giving his own user defined name. For Example the name “Daily Test Execution Status Report” , will be the subject line for emails received from the system to the users. Additionally, the email body may contain useful data about how to correct the fault discovered by the alert.

Setting the time and add the Notifier

A User can set the time to generate the email to be received by a group of Users. The period of time for which an Alert remains active can also be defined, relieving users of having to worry about daily updates.

 

2015-10-24_07-25-34

Tip Of The Week – Build yourself an Aggregate ORCANOS Solution for Cross Projects Reports

October 15th, 2015 Posted by ALM 2.0, KPI, Tip Of The Week 0 thoughts on “Tip Of The Week – Build yourself an Aggregate ORCANOS Solution for Cross Projects Reports”

October 7th, 2015  081c888  Avi Harrary in Tip of the Week, QPack


Start Your Free Trial Now


 

 

This week’s tip addresses how we deal with complex technology such as medical devices, smartphones, or any large scale system. In many cases, we need to provide our executive a well-defined cross-project report that supports their decision process. This is never something that one person should take action on. Instead, such projects are best approached in a collaborative manner.  One of the greatest capabilities ORCANOS has pioneered and brought to the market was the ability to give users a way to simulate the architecture of the product and break it into deliverable components; almost like we do when we build the BOM for our manufacturing process. Using the Solution project work item provides the option to gather under one roof, all the subcomponents needed to manage projects. This type of view in ORCANOS gives you endless ways to review each subcomponent, including all projects details.

 

The example below illustrates my point:

Let’s assume we have 3 different products that are being represented on ORCANOS, using three different ORCANOS Solutions:

  1. Product 1 is being represented by Solution 1, and comprises the following sub-components underneath, which are being represented by ORCNAOS Projects:
    1. Subcomponent 1 is Software Project: Software 1
    2. Subcomponent 2 is a Hardware project: Hardware 1
    3. Subcomponent 3 is a Mechanical project: Mechanical 1
  2. Product 2 is being represented by Solution 2, and includes the following sub-components underneath, which are being represented by ORCANOS Projects:
    1. Subcomponent 1 is Software Project: Software 2
    2. Subcomponent 2 is a Hardware project: Hardware 2
    3. Subcomponent 3 is a Mechanical project: Mechanical 2
  3. Product 3 is being represented by Solution 3, and entails the following sub components underneath, which are being represented by ORCANOS Projects:
    1. Subcomponent 1 is Software Project: Software 3
    2. Subcomponent 2 is a Hardware project: Hardware 3
    3. Subcomponent 3 is a Mechanical project : Mechanical 3

 

Now, let’s assume we would like to have a lateral view, Cross discipline: In This case, we will establish a new QPack Solution called ‘Reports’ , and relate all projects to this Solution. The Reports Solution structure will be as follows:

 

Reports (Solution)

  • Software 1 (Project)
  • Software 2 (Project)
  • Software 3 (Project)
  • Hardware 1 (Project)
  • Hardware 2 (Project)
  • Hardware 3 (Project)
  • Mechanical 1 (Project)
  • Mechanical 2 (Project)
  • Mechanical 3 (Project)

 

This kind of aggregative tree structure, will enable retrieving cross products data, using the ORCANOS Views (Filter) capabilities.

How To Create Cross Project View?

As mentioned above, the solution lies in establishing one ORCANOS Solution, which holds all relevant ORCANOS projects underneath it. From QPack Admin interface you can create a solution such as Portfolio View, which will include as many other project components as required.

  1. Establish a new ORCANOS Solution called ‘Reports’ (How To: http://www.orcanos.com/help/Knowledgebase/creating-a-new-project/)
  2. Relate all projects to this solution (How To: http://www.orcanos.com/help/Knowledgebase/addremove-projects-from-solution/)
  3. Build yourself cross project filters and dashboards.

 

Notice that this technique is effective only when all project data are being managed on a Project level, and not on a Solutions level !!!

Applications:

QMS Solution: This solution includes all the Quality Management System components are useful when required by the Quality Manager to perform tasks. In such cases you may find for example the following projects

  • Document Management System
  • Service Center (Complaint Management)
  • Risk Management (RMF File)
  • QMS E-Forms  (CAPA, ECO, Training, MRB etc.)
  • Risk Mitigation Related Projects (Software, Hardware, Mechanical)

Having all the above projects under one solution gives the QA/RA personnel a means of controlling quality events, and performing all quality control activities as required such as; traceability and action items.

 
2015-10-21_15-37-26

 

OASIS solution for GE Ultrasound multi site production

July 22nd, 2015 Posted by KPI, Orcanos Cafe, Software Lifecycle Management 0 thoughts on “OASIS solution for GE Ultrasound multi site production”

GE Ultrasound success story on the implementation of Orcanos and Polaris joined technology solution for production floor real time reporting and analysis tool.

 

The Blue Ocean Strategy

July 19th, 2015 Posted by KPI, Orcanos Cafe, Software Lifecycle Management 0 thoughts on “The Blue Ocean Strategy”

Elcam Medical Mitigating The Approval Risk – By Placing their documents on Pre-Audit alert system that implemented the complete V&V Medical Device module

July 14th, 2013 Posted by 510(k), Company News, KPI, Software Lifecycle Management, Validation and Verification 0 thoughts on “Elcam Medical Mitigating The Approval Risk – By Placing their documents on Pre-Audit alert system that implemented the complete V&V Medical Device module”

Based 2,323 California biomedical companies research, here are the main 10 threats those companies has reported on

  • #1 – FDA regulatory / environment
  • #3 – R&D productivity
  • #7 – Intellectual property protections
  • #8 – Ability to demonstrate effectiveness
  • #9 – Product liability
  • #10 – Unprepared workforce

The unspoken rule is that at least 50% of the studies published even in top tier academic journals – Science, Nature, CellPNAS, etc… – can’t be repeated. More than that if there was a development behind those studies it was impossible to recreate the documentations requires utilizing their business potential.

According to the same research the following answers given to the question: Why did the company delay the research or development project? Were as follows:

  • 40.2% – Funding not available (Second Round)
  • 27.8% – Regulation (FDA, EPA, SEC)
  • 25.8% – Change in corporate priorities or strategy
  • 4.1% – layoffs
  • 7.2% – Other

Orcanos Implementing NPI (New Product Introduction) for one of the legacy Israeli Medical Device company Elcam Ltd. In this project the focus was on getting the project started on the correct regulatory path and have taken the initiative documents created by the R&D group into a preset system that control and governance the regulatory path selected by the organization. The overall idea was to define the development path in which the specific product shall be using and to match the perfect system that will control and governance each step in the development lifecycle. To achieve this goal we have selected QPack Medical™ system that accepted the validation documents and for each document the system created set of KPI (Key Performance Indicators) as well as KRI (Key Regulatory Indicators) that triggered QPack Medical alert system with Pre-Audit notifications.

 

Mitigating Audit-Submission Risks

 

 

For example:

The insertion of MRD documents during the Idea/Concept stage trggered the following alerts

KPI

  • Market Requirements missing coverage matrix
  • Market Requirements maturity based on functional test results
  • Market Requirements missing due date
  • Unapproved market requirements
  • Readiness for PDR review

 

KRI

  • Missing Market Requirement Document
  • Missing Market Requirement  Specifications
  • Market Requirements missing traceability to product requirements
  • Market Requirements missing validation procedures

 

 

 

 

 

Software Maturity Performance

July 5th, 2013 Posted by KPI, Software Lifecycle Management, Test Management 0 thoughts on “Software Maturity Performance”

IEEE talks about changes made in the software, over 3 dimensions,  to measure the maturity of the software, here in this post we wish to present the relation between test being performed (regardless to their results) vs. the defect discovery during the same period.

 

Test Metrics can be used to measure test coverage prior to software delivery. It provides a measure of the percentage of the software tested at any point during testing.

It is calculated as follows:
Functional Test Coverage = FE/FT
Where,
FE –  is the number of test requirements that are covered by test cases that were executed against the software
FT –  is the total number of test requirements

Software Maturity Metric

Software Maturity Index is that which can be used to determine the readiness for release of a software system. This index is especially useful for assessing release readiness when changes, additions, or deletions are made to existing software systems. It also provides a historical index of the impact of changes. It is calculated as follows:
SMI = Mt – ( Fa + Fc + Fd)/Mt

Where:

SMI is the Software Maturity Index value
Mt is the number of software functions/modules in the current release
Fc is the number of functions/modules that contain changes from the previous release
Fa is the number of functions/modules that contain additions to the previous release
Fd is the number of functions/modules that are deleted from the previous release
Reliability Metrics

Looking at the image below you will find that some of the above calculation can also be reflected in simple presentation of the execution of tests being performed at ALL software releases regardless to the module add/change/delete and at the same time looking at the trend of defects discovery.

Looking at the first period of the software lifecycle you can identify that the number of tests being executed reflects directly on the defect discovery during the same period of time. That is vs. the more advanced period where the software is getting more matured, even those there is dramatic increase in the testing effort, still the defect discovery is not showing the same performance as on the first period.

 

QPack Analytics™ report based on OBIEE technology 

Software_Maturity_Test_Run_vs_Defect_Discovery_Rate

 

 

Orcanos

Contact

8 Tozeret Ha'aretz Street
Tel Aviv, Israel
+972-3-5372561
info@orcanos.com

Copyright © Orcanos, All rights reserved. | Privacy policy | Terms of use