Technoethical assessment with a systematic literature review

Use this framework to perform a technoethical assessment and a systematic literature review for an emerging technology based on Mario Bunge’s technoethics to inform ethical decision making and policy development.

The technoethical framework discussed here is derived from my work on a uOttawa PhD thesis (2020) completed at uOttawa engineering, School of Electrical Engineering and Computer Science (EECS), on the topic of ethical hacking sociotechnology titled Technoethics and sensemaking: Risk assessment and knowledge management of ethical hacking in a sociotechnical society (thesis advisory committee: uOttawa professors Rocci Luppicini, Liam Peyton, and Andre Vellino).

What is a technoethical assessment?
What is a systematic literature review?
Research paper outline (example)

You may also be interested in The technoethics of Mario Bunge.

What is a technoethical assessment?

Mario Bunge’s pragmatic value theory “can serve as a basis for weighing means, goals, and side effects, and thus help to make or adopt rules of conduct both technically feasible and morally right” (1975, p. 76). In Towards a Technoethics, Mario Bunge (1975) suggested three technoethical rules of conduct for the responsible technologist: 1) “To assess a goal, evaluate it jointly with the side effect”; 2) “Match the means to the goal both technically and morally: employ only worthy practical means and optimal knowledge”; and 3) “Eschew any action where the output fails to balance the input, for it is either inefficient or unfair” (p. 78).

Bunge’s (1975/1977) technoethical approach (the three technoethical rules applied within a systems framework) can be applied to assess technology—its use and value within a social system—by weighing the goals (intended ends/potential benefits/opportunities) against the potential side effects (potential costs/risks) with emphasis on efficiency and fairness. A technoethical assessment takes a systems approach to understanding the meanings, ethics, uses and practices, and value (including perceptions of risk) of an emergent technology in society to inform ethical decision making.

What is a systematic literature review?

A systematic literature review is a systematic review of peer-reviewed literature (typically covering the most recent journal articles on a topic) that uses a method called evidence synthesis (science policy or science for policy) to look for answers to a pre-defined question. The purpose of a systematic review is to sum up the best available research on a specific question by synthesizing the findings/results of several studies (grouping the findings/results into themes or categories and making sense of them through critical analysis and/or with the aid of a theoretical framework).

Jesson, Matheson, and Lacey (2011) define a systematic review as “a review with a clear stated purpose, a question, a defined search approach, stating inclusion and exclusion criteria, producing a qualitative appraisal of articles” (p. 12). Systematic reviews document their search strategy so that readers can assess their rigor and the completeness and repeatability of the process. Selection criteria should be decided during the protocol definition stage in order to reduce the likelihood of bias, although they may be refined during the search process.

Here’s a three-minute animated video summary of “the process involved in conducting a systematic literature review” (courtesy of Research Shorts).

“A systematic literature review is one of many methodologies that can be used to conduct secondary research.” (Video by Research Shorts)

Research paper outline (example)

Example (a hypothetical research paper title): A technoethical assessment of AI powered social media OSINT technology applications in business intelligence in the context of privacy and governance

ABSTRACT (~250 words)

The abstract summarizes the background, study purpose, methodology, principal findings/results, practical and theoretical implications of the findings/results, and research significance/contribution to knowledge/scholarship.

Background: the use of technology has exploded in recent years raising concerns about its effect on society and individuals, e.g., effect on privacy
Study purpose: address a gap in literature on on the ethical and social effects of technology…
RQ What are current practices of technology use?
This study/research examined discussions in peer-reviewed journals about the use of OSINT technology from several perspectives…
Method: systematic literature review followed by a technoethical assessment and interdisciplinary synthesis to inform ethical decision making…
Several steps were involved in the collection and analysis of articles in this review.
First, the inclusion criteria specified peer-reviewed journal articles published between 2010 and 2021.
The procedures employed to select relevant studies included both library databases and on-line searches.
The initial search uncovered 50 articles that were reduced to 30 core articles for the final synthesis applying inclusion and exclusion criteria.
Key themes were synthesized according to the area or type of OSINT technology described.
Findings: pointed to a need to regulate OSINT technology…

1. INTRODUCTION (~1 page/500 words)

The topic of the article is clearly stated and why it is relevant to readers/scholars (usually in the introduction section).

2. BACKGROUND (~1 page/500 words)

The research question(s) and study purpose or objective(s) are clearly stated (usually in the background section).

The background section helps readers understand the state of the art and knowledge gaps in the research area that will be addressed by your research, and be prepared to understand the objectives of the research project. To situate your research, show how your study draws on knowledge gained from previous research.

To address a current gap in research focusing on ethical issues connected with OSINT technology use, the following research question guided this study: What is the state of knowledge concerning the use of OSINT technology (technologies) in business intelligence from an ethical perspective? AI powered OSINT technology is an example of an emerging communication/knowledge management technology that raise public concerns regarding privacy and autonomy and create regulatory gaps. OSINT technologies are a good example of how public debate and ethical concerns can influence innovation success.

This research paper has two objectives: First, to summarize the accumulated state of knowledge concerning the use of technology in society. This is accomplished by, (1) identifying the best available work on the topic in an effort to better understand trends in this emerging body of scholarship and suggest areas for future research by summarizing strengths and weaknesses of current research on the topic, predominant research areas/research trends, and knowledge gaps related to technology use and governance in society. The study takes a critical approach to appraising of the state of knowledge about the topic–by recognizing positive as well as negative aspects of a study (especially concerning its methodology) and the study’s contribution to knowledge; and (2) identifying and summarizing key themes related to the use of technology within peer-reviewed academic journals searched from EBSCO, SCOPUS, and Google Scholar and explaining them. Second, to conduct a systemic ethical assessment of the findings (the identified themes) to inform ethical and effective technology use/governance in society.

3. METHODOLOGY

The research method section identifies the research approach (e.g., the qualitative exploratory case study approach) and justifies its suitability for the research question/study objectives, explains the sampling strategy and sampling criteria, and explains the data collection and data analysis procedures.

A systematic literature review was followed by a systems technoethical analysis of the themes (findings) to inform ethical decision making.

A review protocol outlines the seven systematic literature review steps (based on the six stages of a systematic review by Jesson, Matheson, & Lacey, 2011). See Appendix…

Step 1 Define the RQ (the purpose of the review)
“Once a question has been formulated, the research protocol serves as the road map towards its answer” (Jesson, Matheson, & Lacey, 2011, p. 16).

RQ What are current business intelligence applications of OSINT technologies?

Study purpose: to situate AI powered OSINT business intelligence applications within literature, within ethical theories, and within a broader sociocultural perspective to inform decision making/policy making.

Step 2 Identify information sources

Procedures employed to select relevant studies included library and on-line searches. Key databases: Library catalogue, digital libraries, journal databases, official websites, online repositories, bibliographic databases.

Electronic sources with a science and engineering bent used in the review were:
• IEEExplore
• ACM Digital library
• ScienceDirect
• SpringerLink (to access journals such as Empirical Software Engineering and Springer Conference Proceedings)
• SCOPUS (claims to be the largest database of abstracts and citations)
• Google scholar (scholar.google.com)
• Research Gate
• Web of Science

Table A summary of the information sources

Information sources	Pros	Cons
Library catalogue	List the books, journals, and other resources that are held within your library	May not allow you to search for or access their content; too general/broad as a source
Digital library/electronic library	To ensure superior search results search individual databases; use complex search tools available within individual databases to help improve your search results	The databases have been pre-selected by your library staff and the search tools are limited; may not retrieve all published articles in your field of interest
Individual full-text journal databases	Full-text databases provide access to specific articles hosted by content aggregators or individual publishers	The databases may not contain the most recent issues of the journals you are interested in; the databases may not contain all the relevant journals for your field
Official websites
Online repositories
Bibliographic databases	Cover the content of many publishers, e.g., The Journal Citation Reports within Web of Knowledge provides information about the most frequently cited journals and the highest impact journals in a field

Step 3 Do literature search (keyword search)

“Systematic reviews are based on a defined search strategy that aims to detect as much of the relevant literature as possible” (Jesson, Matheson, & Lacey, 2011). Develop and follow a search strategy in consultation with librarians.

(1) Identifying/selecting keywords for the research topic:
• select words from your research statement or research question
• identify similar and related words, e.g., synonyms, broader terms or narrower terms
• identify keywords and subject terms from the databases you are searching by browsing the online subject terms
• you can use word combinations as keywords

(2) Keyword search strategy/creating search lists/search strings:
• search the Abstracts for keywords
• use the Boolean operators AND/OR/NOT to improve your search results
• some databases will have a wild card such as an* that can be used to truncate words
• many databases have a facility to search for words in proximity of each other, e.g., in the same sentence or within one or two words of each other

Example: Create about 4 OR lists; link the four OR lists using the Boolean operator AND to construct the search strings such that the search strings are appropriately derived from the research questions:
hacker OR hacking OR ethical hacker OR ethical hacking
AND
ethics OR ethical OR ethical theories OR ethical theory OR normative theory OR normative theories OR moral value OR moral theory … OR ethic* OR ethic* theor* … and so on.

Table Search record for handling search results

Data source	Date of search	Search strings (“AND” “OR” “NOT”)	Search limiters (e.g. years covered)	Relevant results with search strings (key references)	Literature with selection criteria; notes	Search parameters
SCOPUS
IEEExplore	26 Nov 2020	Full text and metadata				Article Title, Abstract, Keywords
Google Scholar
EBSCO
PsychINFO

Table Breakdown of top 30 journals publishing on topic

Journal title	Frequency	Percent
Journal of.. Journal of…

Step 4 Apply inclusion/exclusion criteria

After the literature search you will need to read the prospective articles and decide which are appropriate for inclusion in your review according to your protocol criteria. First, read the title, the abstract, and maybe the introduction and conclusion sections of articles. Then, screen the papers in their entirety for the key information that you will need for your data extraction.

Practical screen–screening for inclusion, this step requires that the reviewer be explicit about what studies were considered for review, and which ones were eliminated without further examination.

Quality appraisal–screening for exclusion, the reviewer needs to explicitly spell out the criteria for judging which articles were deemed inadequate for retaining/inclusion in the final synthesis. All included articles need to be scored for their quality, depending on the research methodologies employed by the articles.

Table Inclusion and exclusion/selection table

Search strategy identified 50 publications involving the topic
Studies included in further analysis: 40	Studies excluded and reasons for exclusion: 10

Step 5 Data/information extraction from the included studies for thematic analysis

[Analysis =data extraction (Step 5), quality assessment (Step 6), and Synthesis (Step 7)]

After the studies to be included in the review have been identified, the reviewers need to systematically extract the applicable information from each study (that addresses the research questions) to serve as the raw material for the synthesis stage. A data extraction strategy (as part of the review protocol) specifies what/how the information required from each primary study will be obtained, e.g.:
Data extraction elements:
Author and publication details (bibliographic details)
Paradigm (academic discipline)
Aim and focus of the paper
Method details
Theory or models
Data about technology meanings, uses and practices, value

Step 6 Quality assessment (literature appraisal)

Assessing the quality of the research so far selected for inclusion in the review against a checklist/quality assessment criteria. Study the texts in chronological order.

Table Articles excluded from the systematic review and reasons

Reference	Aim of research	Focus and location	Method	Reasons for exclusion
Ref1 Ref2 Ref2 etc

Step 7 Synthesis (of the extracted data)

Synthesis on thematic basis pools the evidence from single studies. “The aim is to collate and present the extracted data from primary studies so that the characteristics and results of the study are summarized” (Jesson, Matheson, & Lacey, 2011, p. 123). “In the data extraction stage you unpacked each article.” In the synthesis stage “you have to put them all together again, but this time telling a new story or making new connections. This is your contribution to knowledge, or filling the knowledge gap” (p. 123).

4. FINDINGS AND ANALYSIS

4.1. General findings and analysis

Table 1 Iteration of search operations with refined search terms
Table 2 Distribution of constructs
Table 3 Technoethics of OSINT technology use

Table 3 presents key perspectives as observed through the lens of a technoethical inquiry that warrant consideration when attempting to assess the ethical merits of OSINT technology use.

4.2. Analysis of broad themes/constructs/key perspectives

Table 3 illustrates key perspectives (historical, theoretical, political, legal, economic, sociocultural, stakeholders, intended ends, possible side effects, and means) that warrant consideration when attempting to arrive at an ethical assessment when OSINT technology is used commercially.

Table 3 Technoethics of OSINT technology use (hypothetical constructs)
Theoretical perspectives
Necessary consideration of individual ethics
Need to readjust organizational ethics in conjunction with individual ethics
Need for societal laws reflecting the new trend of using OSINT in BI
Lack of information about undesirable events, increasing ethical risk
Historical perspectives
Development from innovative technology
Original non-commercial use by hobbyists, but in a confined location
Political perspectives
Using OSINT in marketing activities
Selling OSINT data to political actors
Sociocultural perspectives
Privacy infringement
Danger of reselling private data
Leveraging the positive impacts on our daily lives with new technology
Economic perspectives
Stimulation of new business trends
Fast deployment of advertising material and delivery of merchandise
Good return of investment
Stakeholders perspectives
Consideration of OSINT use in other applications by large corporations, e.g. Amazon
Need for the creation of laws and regulations
Need for more ethical awareness programs for commercial use of OSINT technologies
Intended ends and possible side effects
U.S. government ban on similar technologies
Public concerns about privacy and unethical actions
AI and effects on autonomy
Means and intended ends
Enforcement of laws
Licensing of OSINT technologies
Awareness program for commercial marketers who use OSINT
Overall value in terms of efficiency and fairness
Enabling effective delivery of advertising messages and small packages
Leveraging of technological innovation
Establishment of operational rules for commercially used OSINT technologies

5. DISCUSSION

This study applied the technoethical approach, wherein the ethical aspects of technology and how technology shapes a society are studied by assessing ethical uses of technology in order to influence technological development and improve daily life in a society.

Based on the general findings, the broad themes/constructs were grouped into three themes for more in-depth analysis: political economic autonomy as privacy, security as privacy, and trust.