2.1.

Participants

The experimental procedure conformed to the Declaration of Helsinki and was approved by the local ethics committee [Ethics Review Committee Psychology and Neuroscience (ERCPN)]. Ten healthy individuals [10 females, 9 right-handed, age = 26.0 ± 7.2 years (mean ± SD), all with reportedly normal hearing; see Table 1] were externally recruited or were recruited from students and staff members of the Faculty of Psychology and Neuroscience at Maastricht University. Written informed consent was acquired prior to the experiment. Participants were compensated financially for their participation (7.50 €/h in form of vouchers).

Table 1

Participant demographics.

2.2.

General Procedure

The study was composed of a training session and the fNIRS-BCI session (see Fig. 1). The training session entailed a training of the mental imagery task and familiarization with the communication paradigm. To evoke a measurable signal, a combined motor-speech imagery task was used. Participants were asked to imagine checking off a list with their right hand while saying “check” or “correct” simultaneously for $\sim 2\mathrm{s}$. Each participant was instructed on how to perform the mental task and trained to encode answers until they felt comfortable in doing so (for $\sim 15\min$). Additionally, participants’ head circumference was measured to select an appropriate cap size. Cap sizes used in this experiment ranged from 56 to 58 cm (see Table 1). After training, participants chose 10 unobtrusive binary questions to be answered during the BCI session from a list of 45 possible questions (see Appendix 1 in the Supplementary Material). To ensure an equal distribution of both answer options regardless of choice, each question was to be answered truthfully first before also encoding the opposite answer. Finally, to quantify participants’ suitability for fNIRS measurements, a suitability score was determined by means of an in-house questionnaire (see Appendix 2 in the Supplementary Material) completed prior to the measurements.²⁵

Fig. 1

General procedure. Each participant underwent a training session (day 1) comprising of assessing fNIRS suitability, determining the correct cap size, and familiarization with the mental-imagery task and encoding paradigm. In the experimental session (day 2) two localizers were performed to determine a COI followed by 10 encoding runs to encode answers and decode the corresponding signal online. Subjective ratings were acquired after each run and finally the mental-imagery task was rated at the end of the session.

During the second part of the study, the fNIRS-communication paradigm was tested. In the beginning, participants were seated in a sound-attenuating and darkened room. Nasion-inion distance was measured to ensure correct positioning of the cap. FNIRS optodes were additionally covered by a black cap to prevent influence of external light on the measurements. Finally, the room was equipped with a microphone and speakers to enable communication between participant and researcher and to play recorded auditory instructions given during the experiment. Participants were instructed to get into a comfortable position and to relax their jaw and facial muscles during the session.

The measurements commenced with two localizer runs to determine the most informative channel [channel of interest (COI)]. After the first localizer, participants were able to take a short break during which the channel setup was changed for the second localizer. Thus, each participant underwent two localizer runs with a different setup per run. To counterbalance possible effects of fatigue in the setup choice, half the participants started with one setup and the remaining half with the other one. Following the localizers, 10 encoding runs (1 per question) were performed. For encoding runs, the optode setup, including the most informative channel was chosen for each participant. In between runs, participants were allowed to take a short break, adjust posture, and drink water. After each run, they were asked to rate their current comfortability, concentration, and alertness on a scale between 1 (not at all) and 10 (very). After the final run, participants were asked to rate the easiness and pleasantness of the mental task.

2.3.

Communication Paradigm

Participants answered binary questions by performing a mental-imagery task for $\sim 2\mathrm{s}$ at different points in time to indicate the answer. Simple auditory instructions, i.e., concise spoken commands (“yes,” “no,” and “switch”) informed participants of the start of each encoding window. Note that no explicit instruction to stop the mental imagery was required as participants had been trained to perform the task for $\sim 2\mathrm{s}$. Instructions and corresponding time point triggers were implemented using the stimulation software NIRStim 4.0 (NIRx Medizintechnik GmbH, Berlin, Germany). Each run consisted of a 40 s baseline [to calculate oxyhemoglobin (HbO) and deoxyhemoglobin (HbR)] concentrations preceding 10 trials in which a question was answered 10 times in total (5:08 min/run; see Fig. 2). Each trial consisted of a 5-s pre-window, a 2-s window for one encoding option (cued by auditory instructions) followed by 8 s resting time before the second 2-s window (cued by auditory instructions), and then a 10-s post-window. In the first five trials, answers should be given truthfully and in the last five the opposite answer should be encoded. An auditory cue “switch” instructed participants when to change their encoded answer. This ensured that each answer option was encoded equally often independent of a participant’s choice of questions and answers. To indicate the answer, the participant had to perform the mental imagery task for about 2 s at the respective time of a trial. Each trial was initiated by the first auditory cue for indicating “yes.” If the participant wished to answer “yes,” they then performed the 2-s mental imagery. 10 s after the first auditory cue a second cue for “no” followed. To indicate the second answer option, the participant performed the task in the respective time window.

Fig. 2

Answer encoding scheme. The figure shows the two possible answer-encoding scenarios for both a “yes” (top) and “no” (bottom) answer. Expected oxyhemoglobin (HbO) time courses (black curves) in mental imagery-related brain regions are displayed. When participants intended to encode “yes,” they performed combined motor-speech imagery causing HbO to rise during the encoding time span for “yes” (2s; green bars). When participants wished to encode “no,” they performed combined motor-speech imagery causing HbO to rise during the encoding time span for “no” (2s; red bars). Participants encoded the same answer five times per run before a designated switch (auditory cue “switch”) indicated to encode the opposite answer for five times again.

2.4.

FNIRS Suitability Questionnaire

Because fNIRS constitutes an optical neuroimaging method, its signal quality can be crucially impacted by the presence of certain physical features of participants, such as hair characteristics (e.g., color, thickness, and density^42–47), skin pigmentation,⁴⁷ and head size (inter-optode distance and skull thickness variation affects light absorption). To quantify participants’ suitability for fNIRS measurements, a suitability score was determined by means of an in-house questionnaire (see Appendix 2 in the Supplementary Material) completed prior to the measurements.²⁵ The physical features of interest were rated on a scale ranging from 0 to 4 regarding their risk to absorb NIR light. Individual ratings were summed up to a total score (possible scoring range: 1 to 21). A low fNIRS-suitability score has been associated with improved signal quality.²⁵

2.5.

Data Acquisition

Hemodynamic brain activity was measured with an NIRScout-816 system (NIRx Medizintechnik GmbH, Berlin, Germany). Eight detector and nine source optodes (small light emitting diodes were used to produce light at wavelengths of both 760 and 850 nm. Recorded signals were sampled at a rate of 6.94 Hz. Optodes were positioned according to the international 10 to 20 EEG system. The optodes always covered the left-hemispheric fronto-temporo-parietal cortex for optimal coverage of brain regions activated during the performance of the combined motor-speech imagery.⁴⁸

To increase the chance of defining a promising channel-by-chromophore combination for each participant for the subsequent encoding runs, two alternative setups were used in two initial localizer runs. For one setup, optodes were positioned in a checkerboard pattern, interleaving sources with detectors. For the other setup, positions of sources and detectors were changed thereby forming a rowed placement with rows of sources alternating with rows of detectors. Thus, one setup partially covered those cortical regions that were not covered by the alternative one (see Fig. 3).

Fig. 3

3D view of implemented fNIRS optode setups. “Checkerboard” (a) and “rowed” (b) setup were used during the initial localizer procedure. Light absorption was measured using nine sources (S; red) and eight detectors (D; blue) placed over the left-hemispheric (pre)motor and speech-related regions. For the “checkerboard” setup, sources were positioned on FC5 (1), C3 (2), CP5 (3), P3 (4), FC1 (5), Cz (6), CP1 (7), Pz (8), and 1 cm posterior of FCz (9; SDC source) while detectors were placed on F5 (1), C5 (2), FC3 (3), CP3 (4), FCz (5), C1 (6), CPz (7), and P1 (8). The average channel distance was 38.26 ± 3.5 mm (not including the SDC). For the “rowed” setup, sources were positioned on F5 (1), C3 (2), C5 (3), P3 (4), C1 (5), Cz (6), P1 (7), Pz (8), and 1 cm posterior of FCz (9; SDC source) while detectors were placed on FC5 (1), CP5 (2), FC3 (3), CP3 (4), FCz (5), FC1 (6), CPz (7), and CP1 (8). The average channel distance was $47.04\pm 8.4\mathrm{mm}$ (not including the SDC). In total, the “checkerboard” setup contained 25 channels and the “rowed” setup contained 32 channels, including an SDC (S9-D5). The 3D representation was created with NIRSite v1.0 software (NIRx Medizintechnik GmbH, Berlin, Germany).

A custom-build short-distance channel (SDC) was created by placing source S9 as close as the optodes would allow ($\sim 10\mathrm{mm}$ away) to detector D5 on the same sagittal plane that connects D5 to source S6 (see Fig. 3). The mid-sagittal sinus covered by the added SDC and other large vascular structures⁴⁹ have been shown to correlate with low frequency oscillations and cardiac signals.⁵⁰ The SDC was used in both setups to acquire these signal influences and thereby account for the (extracerebral) physiological noise in the region covered by the setups.³⁰ In total, the setups contained 25 (“checkerboard”) and 32 (“rowed”) direct-neighbor channels including the SDC, respectively.

2.6.

Localizer Runs

Two localizer runs were performed for each participant to assess the strength of their hemodynamic responses and determine a channel most sensitive to the answer decoding. Each run consisted of 20 trials of 2 s each during which participants were instructed to encode “yes” for the first half of the trials followed by encoding “no” for the subsequent trials. An auditory spoken cue “switch” indicated to the participants when to alter their answer encoding. After the first localizer participants were able to take a break during which the channel setup was changed to another one for the second localizer. Thus, each participant underwent two localizer runs with a different setup per run. The first participant tested (here referenced as P07) performed 30 trials in total instead of 20 during the localizer but did not differ in number of trials during subsequent encoding runs.

2.7.

Data Analysis

For data analysis, the software Turbo-Satori (v.1.2.8, Brain Innovation B. V., Maastricht, The Netherlands) was used.⁵¹ Measured time course values were converted to HbO and HbR values using the modified Beer–Lambert law. Moving-average high-pass and low-pass filters were applied (high-pass cut-off: 0.010 Hz, filter order: 1; low-pass cut-off: 0.250 Hz, filter order: 2). Data were analyzed online (i.e., in real-time) and offline using univariate general linear model (GLM) analysis.

Selection of an ideal channel and Hb-type for each participant was based on the localizer data. Specifically, predictors for each answer option were convolved with a standard hemodynamic response function (HRF; double Gamma function, the onset of response and undershoot was 6 and 16 s, respectively, dispersion 1 s, response to undershot ratio 6). A single most informative COI and corresponding most informative Hb-type was determined based on the chromophore and channel that led to the highest $t$-statistic of the active encoding versus rest contrast across localizer runs. Further, the SDC time course was used as a confound predictor to regress out noise. Localizer data were additionally used to create topographical maps of HbO and HbR activation changes obtained for a group fixed effects analysis (with $n=10$, Bonferroni-corrected $p<0.01$) during combined motor-speech imagery task for the setups using Satori (version 1.8). During answer-encoding runs, the encoded answer per run/question was decoded online by fitting a GLM to all 10 trials in each run. Two answer predictors were each convolved with an HRF to model all 10 trials. $T$-values for each predictor were determined based on the previously chosen COI and corresponding chromophore. The answer that led to the highest $t$-estimate of the contrast “yes” versus “no” was considered the selected option. From these answers, a whole-run accuracy was acquired for each participant by dividing the correctly encoded number of runs by the total number of performed runs. In addition, the SDC time course was used as confound predictor again. Subsequently, the data of the encoding runs of each participant were analyzed using offline GLM analysis to calculate a single-trial (ST) accuracy per participant. Mean accuracies were evaluated in a confusion matrix per participant using a ${\chi }^2$ test to identify participants who performed significantly better than theoretical chance level (= 50%) on ST level. Due to data corruption, one encoding run was lost for one participant (P07).

The effect of the number of trial repetitions on mean accuracy on individual and group levels was systematically evaluated post hoc.³⁰ Answers were decoded anew from a reduced number of trial repetitions following the same approach as described above. Concretely, the accuracies of all consecutive trial combinations for every trial number ($1:n$ trials, where $n=[\mathrm{1,2},\mathrm{3,4},\mathrm{5,6},\mathrm{7,8},\mathrm{9,10}]$) were computed. For example, to obtain the accuracy of eight trials, trial combinations 1-2-3-4-5-6-7-8, 2-3-4-5-6-7-8-9, and 3-4-5-6-7-8-9-10 were used. The effect of number of repetitions on the decoding accuracy at the group level was quantified with Spearman’s rho correlation coefficient. The effect of number of trials was additionally evaluated using information transfer rate (ITR), defined as in Allison, Dunne⁵²

3.1.

Channel Selection

Overall, 10 participants took part in the experiment (see Table 1). For all participants, an individual most-informative COI was determined using a localizer procedure described above (see Table 2 for details). Overall, COIs were mostly located above dorsal and ventral premotor area [see Figs. 4(a) and 4(c)]. For two individuals, most reliable activation was detected above superior parietal lobe and precuneus [see Fig. 4(a)]. Generally, the highest $t$-value was found for changes in HbR concentrations in all participants. The optode-setup choice based on the most-informative COI resulted in the “rowed” setup for 6 out of 10 the participants (see Table 2). The average distance of all COIs was $41.46\pm 6.1\mathrm{mm}$ as provided by NIRSite ($39\pm 3.3\mathrm{mm}$ for “checkerboard” only; $43.1\pm 7.1\mathrm{mm}$ for “rowed” only; for more details, see Table S5 in the Supplementary Material).

Table 2

Setup choice for each participant based on channel selection.

Fig. 4

Results of the localizer procedure. (a) Selected COIs for “checkerboard” (arrows) and “rowed” (dashed arrows) setup are marked. Sources (gray) and detectors (white) for both setups are displayed. (b) Topographical maps of HbO (red) and HbR (blue) activation change obtained by group fixed effects analysis (with $n=10$, Bonferroni-corrected $p<0.01$) during combined motor-speech imagery task for the “checkerboard” setup. Key speech-imagery (dashed arrows) and right-hand motor-imagery (arrows) areas are marked. Due to the limited spatial resolution of fNIRS, the labeling of the brain regions should be considered an approximation. vPM, ventral premotor area; SMA, supplemental motor area; dPM, dorsal premotor area; M1, primary motor cortex; S1, primary sensory cortex; and S2, secondary somatosensory cortex. Due to an error during the experimental session, a non-ideal setup was chosen for this participant (“checkerboard” instead of “rowed” setup).

3.2.

Communication Accuracy

During the sessions, answers could be encoded in real-time using a whole-run online GLM. An average accuracy of 85.78% was obtained across participants with a minimum of 40% (P01) and a maximum of 100% (P03 to P06, P09). All participants except two (P01 and P08) showed an average whole-run accuracy $>70\%$ [see Fig. 5(a)]. To speak of a robust communication method, a communication accuracy of more than 70% is considered sufficient in a two-choice paradigm with a sample size equal to 10.^53,54

Fig. 5

Communication accuracies at individual and group level. Whole-run accuracy (a) and single-trial accuracy (b) are plotted for each participant (sorted by single-trial accuracies) and on group level ($x$-axis). Standard deviations of group averages are indicated by black lines. Borders of bars represent the individual setup choice. FNIRS-suitability scores are indicated in the single-trial accuracy graph (black dots). The gray areas mark whole-run accuracies $\ge 70\%$ (a) and single-trial accuracies that were significantly above chance level as determined by a ${\chi }^2$ test (with a corresponding $p\le 0.05$; panel b). Individual easiness/pleasantness ratings of the mental task are provided.

Using an offline GLM analysis, individual answer trials of participants could be decoded correctly with an average accuracy of 68.09% (theoretical chance level: 50%). ST accuracies varied overall from 49% (P01 and P08) to 91% (P03). In 6 out of 10 participants, ST accuracies were significantly above chance level as calculated with a ${\chi }^2$ test [see Fig. 5(b)]. Average decoding accuracy of the first five trials (true answer encoded) within an encoding run was $67.0\pm 0.15\%$, whereas it was $69.2\%\pm 0.14\%$ for the second half (opposite answer encoded). Average accuracies of trials of “yes” or “no” encodings were $68.4\%\pm 0.13\%$ and $67.8\%\pm 0.15\%$, respectively (see Table S4 in the Supplementary Material). Finally, ST accuracies correlated significantly with fNIRS-suitability scores ($r=-0.67$, $p=.03$) and localizer t-values within COI ($r=0.71$, $p=0.02$). Subjective ratings concerning experienced comfortability, alertness, and concentration (see Fig. 7) did not correlate with ST accuracies. Note that these correlations and their significance should be interpreted with caution as the number of participants was not sufficiently high to draw a firm conclusion.

Fig. 7

Average subjective ratings. Average ratings of concentration, alertness, and comfortability are plotted for all runs (and corresponding measurement time). Standard errors are indicated by bars.

3.3.

Effects of Number of Trial Repetitions

The assessment of the effect of the number of trial repetitions on decoding accuracy showed that a mean accuracy $>70\%$ could have been achieved already when averaging the data of two trials ($\mu =74.15\%$, $\mathrm{SD}=15.67\%$, and $M_x=73.33\%$) and an accuracy of $>80\%$ when considering data of four trials ($\mu =81.76\%$, $\mathrm{SD}=17.69\%$, and $M_x=87.86\%$). The average accuracy reaches the maximum of 87% after nine trial repetitions and then stagnates [see Fig. 6(a)]. A significant positive correlation between the accuracies and the number of repetitions was found as calculated with Spearman’s rho correlation coefficient ($\rho =.997$, $p<0.001$). Moreover, the ITR computation indicates that the highest ITR values can be reached, on average, when using four trials ($0.13\text{bits}/\min$) instead of 10 [$0.09\text{bits}/\min$; see Fig. 6(b)].

Fig. 6

Effect of the number of trial repetitions on decoding accuracy. (a) The box-plot shading depicts classification accuracy for all numbers of repetitions used for decoding: from ten trials to a single trial. Mean values (horizontal lines) and ITR (black bars) are indicated for each number of trials. Participant-wise accuracies (%) are plotted for each number of trial repetitions (colored dots), and average participant-wise accuracy is represented through colors ranging from green to red. The dashed line shows the chance level. The number of trials used to decode each run influences the decoding process. Mean- and several single-participant accuracies remain above chance level even on a single-trial level. (b) Average (colored markers) and single-participant (gray markers) ITR values (bits/min) for different numbers of trial repetitions (curves) as a function of achieved classification accuracies. Lines represent the theoretical values the ITR can take (ranging from 0 to 1) as a function of the number of classes, trial duration and accuracy based on Eq. (1).

3.4.

Subjective Ratings

Subjective ratings of comfortability, alertness, and concentration remained high overall (concentration: $\mu =7.91\pm 0.95$; alertness: $\mu =7.64\pm 1.14$; comfortability: $\mu =7.53\pm 1.26$; see Fig. 7). They ranged from 5 (P02) to 10 (P04) for concentration, 4 (P05) to 8 (P04) for alertness, and 4 (P05/P10) to 10 (P04/P08) for comfortability (see Tables S1–S3 in the Supplementary Material). A decrease of all subject ratings could be observed from the first to last run. The steepest decrease, while still mediocre, was observed in comfortability (from $8.3\pm 1.19$ in the first to $6.7\pm 1.79$ in the last run). The task was judged as pleasant and easy (pleasantness: $\mu =8.35\pm 1.61$; easiness: $\mu =8.9\pm 1.14$; see Fig. 7). Comparing ratings and accuracy, no obvious trend was observable.

4.1.

Encoding Period of 2 s Enables Effective and Convenient Communication

To advance progress towards an efficient and convenient binary fNIRS-BCI, a communication paradigm exploiting temporal fNIRS-signal features and using an encoding time of only 2 s was tested. Whole-run communication accuracies were decoded online. On average, a high online communication accuracy was obtained ($\mu =85.78\%$). While communication accuracy varied inter-individually, a sufficient communication accuracy ($>70\%$) was obtained for the vast majority (80%) of the tested healthy participants. Only two participants (P01 and P08) obtained communication accuracies that did not exceed the 70% criterion considered necessary for robust 2-class communication [see Fig. 5(a)].^53,54

The data of the communication runs were further analyzed offline to obtain individual ST accuracies and a group mean. Overall, a high ST accuracy (68.09%) was achieved with 6 out of 10 participants showing a significant individual ST accuracy. Expectably, only chance-level ST accuracies were obtained for those two participants for whom insufficient communication accuracies were observed online [see Fig. 5(b)]. By comparison, other studies employing a binary communication approach obtained ST decoding accuracies of 64.4%³⁶ and 77.2%.³⁴ Communication accuracy obtained when combining several trials reached comparable results when decoded in real-time. One study, for instance, obtained 67% accuracy when combining 20 trials in real-time analysis.²⁵ Another study reached 77.4% online accuracy combining three 20 s-trials after training a classifier offline.³⁹ Offline analysis of multiple combined trials generally yields better results. A study combining 20 trials of 15 s-encoding windows each obtained an offline whole-run accuracy of 80%.⁹ Accuracy went up as high as 89% in a study by Sitaram et al.² utilizing 20 trials with 10 s-encoding windows per mental imagery task.

Notably, the ITR of the paradigm is low for any number of trial repetitions compared to more advanced communication BCIs.^30,55 This is partly due to the length of the pre- and post-window of trials and the small number of answer options, which both yields lower ITRs regardless of accuracy compared to more complex paradigms. The calculation of the ITR was useful in showing the potential for reducing the number of trial repetitions needed for robust communication in some participants. Regardless, the paradigm’s value lies not in increasing the ITR, instead its focus was the decrease of effort for participants through a shorter encoding time. Results indicate that a mental imagery task of 2-s duration is sufficient for most participants to answer binary questions accurately.

Overall, results show that encoding windows as short as 2 s yield comparable accuracies both on an ST and multi-trial level of analysis. Importantly, most participants (7/10) could have reached sufficient accuracy when considering as few as four or even fewer trials as shown during offline analysis, suggesting that speed of communication can be further increased. Note however, that for some individuals (P01, P02, and P08), a larger number of trials encoding one answer or a longer encoding period might have been beneficial and could result in robust communication accuracy (see Fig. 6).

4.2.

Most Informative Channels over the Prefrontal Motor Areas

It was expected that activation caused by the combined motor-speech imagery would be reflected in channels directly above the prefrontal motor areas. In line with this, the most informative channel was located above these areas in most of the participants (8/10). For two pairs of participants, the same channels were chosen as COI (S2-D3 and S2-D5, respectively, see Table 2). While the combinatory use of the two different optode setups (“checkerboard” and “rowed”) led to a longer experimental session and more effort for the participants (due to performing an additional localizer), it was a pragmatic approach to improve coverage of the fronto-temporo-parietal cortex. Results demonstrated that the COI was found almost equally often in the “checkerboard” and the “rowed” setup (see Table 2, Fig. 3) suggesting that using different setups might be useful. Note, however, that the present study was not intended (and is not suited) to systematically evaluate potential benefits of such an approach. This methodological aspect is currently investigated in a separate study.

Due to an experimenter error, the wrong setup (the “checkerboard” setup) was used during the encoding runs for one participant (P08). The most-informative COI in the ideal (“rowed”) setup (S3-D1) for this participant was actually located above the prefrontal motor areas whereas the used COI (S8-D7) was located more posterior above the parietal cortex. The fact that the premotor area was selected for use in the encoding runs for almost all other participants, suggests that this difference in location could (at least partially) explain why decoding results for P08 were low. Notably, the selected COI of the other participant with chance-level accuracy (S8-D8, P01) was also located more caudally although the signal quality of this participant was generally weak, and the location overlap here might be just coincidental. It should be noted that only one SDC was included in each setup although the use of multiple SDCs per setup would be ideal to account for the heterogeneity in scalp hemodynamics when covering several brain regions.⁵⁶ While this might make application of setups in a natural (non-lab) environment easier, future studies should investigate whether signal quality can be significantly improved through usage of SDCs on multiple sites. Furthermore, a potential research question is whether participants have individual ideal locations for SDCs. If only one SDC is available, it would be ideal to place the SDC accordingly.

4.3.

On the Benefits of Using a Combined Motor-Speech Imagery Task and the Association of fNIRS Suitability and BCI Performance

The purpose of using a combined motor-speech imagery task was to increase the amount of cortex activated by mental-task performance and therewith to boost the chance to define a promising channel in each participant. After the last encoding run, participants were asked to rate the easiness and pleasantness of the mental task. The task was rated easy and pleasant on average, suggesting that it required only little cognitive effort (see Fig. 7). Since the present study was meant to assess feasibility of shortened encoding times rather than cognitive workload, simple subjective ratings were collected to confirm that the combination of motor and speech imagery was not too demanding for participants. A more detailed systematic evaluation of cognitive workload in relation to the paradigm and imagery could be done in future work. Additionally, one should systematically investigate whether an increased number of answer options could result in high cognitive demand despite short encoding time.

Moreover, participants were asked to rate their current comfortability, concentration, and alertness on a scale between 1 (not at all) and 10 (very) after each run. Ratings of concentration, alertness, and comfortability during the experimental session remained generally high. A small decrease could be observed over runs, particularly for comfortability (see Fig. 7 and Tables S3–S5 in the Supplementary Material). Note that the measurements took place in a darkened room with closed door to improve signal quality and to keep the environment constant across participants. A more natural measurement setting might improve user comfort. To quantify participants’ suitability for measurements, an fNIRS suitability score was determined for each participant by means of a questionnaire filled out prior to the experimental session (see Appendix 2 in the Supplementary Material). ST accuracies correlated negatively with the fNIRS-suitability scores indicating high scores (inferior suitability) can be predictive of limited accuracy [see Fig. 5(b)]. It should be noted that the participant with the lowest accuracy (P01) also had a quite high fNIRS suitability score (14/21 possible points) indicating low fNIRS suitability in terms of hair texture, hair and skin color, and/or head size. While the score offers the possibility to predict a participant’s BCI performance to some degree, obtained communication accuracies also show that participants can still yield great accuracy despite a suboptimal fNIRS-suitability score (e.g., P06). Thus, while the score may be used as an initial judgement method of general fNIRS suitability, it is not the only factor that determines BCI performance. Limited fNIRS suitability might be compensated by other factors, such as motivation or mental-task performance.

4.4.

Remaining Limitations

The present study has some limitations that should be considered. Coincidentally, only female participants were included; in addition, the number of participants is limited ($n=10$) although in BCI experiments (where the focus is on the individual level), the current sample size is sufficient to demonstrate feasibility of the paradigm at hand.^2–4,7,9 A replication of the study, preferably performed in another lab, with a larger number of participants, including male participants would be a valuable extension.

While the overall aim was to design a paradigm aiding everyday use of an fNIRS-based communication BCI, the conditions under which answers were encoded, were kept as constant across participants as possible. However, while a darkened, closed-off room likely aids consistency and signal quality, it does not represent a realistic situation of potential application in patients. Thus, to conclusively proof suitability of fNIRS for everyday use, this BCI-communication method should also be tested in a less controlled environment, i.e., natural environments, as was done recently in a study by Nagels-Coune et al.²⁷

Finally, the design of one communication trial resulted in a shorter resting period following a “yes” compared to a “no” answer which could potentially induce a bias. However, this difference did not confound the present results as there was no difference in encoding accuracies between “yes” and “no” answers (see Table S4 in the Supplementary Material).

4.5.

Future Research Directions

The presented fNIRS communication BCI opens up several new directions for further investigation. First, the present study used a questionnaire score to determine suitability of participants for fNIRS, and even though obtained scores seem to correlate with ST decoding accuracy, this measurement is based on the subjective evaluation of participants’ physical appearance. The presented FNIRS findings suggest that the signal quality observed during localizer runs could also be indicative of attainable communication accuracy. Future studies should investigate whether one could determine fNIRS suitability of a BCI user based on localizer data alone. Second, the chosen combined motor-speech imagery task was thought to be easy in terms of cognitive effort while also increasing the chance to find a suitable COI in all participants. To further improve the signal quality for each participant individually, one could choose an individually selected, optimal task per user which elicits the best possible signal. Finally, results (see Fig. 6) indicate that, at least for certain participants, the number of trials required for robust answer decoding can be reduced. It remains to be tested whether it is possible to predict the number of required trials for an individual to adjust run time accordingly and then gain sufficient accuracy for all tested participants who generally are suitable for fNIRS.