Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance importActorProfile function to normalize file names, skip sys… #8

Merged
merged 1 commit into from
Feb 8, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 32 additions & 6 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@ import * as tar from 'tar-stream'
import { type Pack } from 'tar-stream'
import YAML from 'yaml'
import { type Readable } from 'stream'
import path from 'path'

export interface ActorProfileOptions {
actorProfile?: any
@@ -186,12 +187,26 @@ export async function exportActorProfile({
export async function importActorProfile(
tarStream: Readable
): Promise<Record<string, any>> {
console.log('🚀 Starting to process tar stream...')
const extract = tar.extract()
const result: Record<string, any> = {}

return await new Promise((resolve, reject) => {
extract.on('entry', (header, stream, next) => {
const fileName = header.name
// Normalize fileName to include only `activitypub/filename`
const originalFileName = header.name
const fileName = `activitypub/${path.basename(originalFileName)}`

// Skip system-generated files
if (
fileName.startsWith('activitypub/._') ||
fileName.endsWith('.DS_Store')
) {
console.warn(`Skipping system-generated file: ${fileName}`)
next()
}

console.log(`Processing file: ${fileName}`)
let content = ''

stream.on('data', (chunk) => {
@@ -202,31 +217,42 @@ export async function importActorProfile(
try {
if (fileName.endsWith('.json')) {
result[fileName] = JSON.parse(content)
console.log('Parsed JSON file successfully:', fileName)
} else if (fileName.endsWith('.yaml') || fileName.endsWith('.yml')) {
result[fileName] = YAML.parse(content)
} else if (fileName.endsWith('.csv')) {
result[fileName] = content
} else {
console.warn(`Unsupported file type: ${fileName}, skipping...`)
}
} catch (error: any) {
reject(new Error(`Error processing file ${fileName}: ${error}`))
console.error(`Error processing file ${fileName}:`, error.message)
} finally {
next() // Always continue
}
next()
})

stream.on('error', (error: any) => {
reject(new Error(`Stream error on file ${fileName}: ${error}`))
console.error(`Stream error on file ${fileName}:`, error.message)
next() // Continue even on stream error
})
})

extract.on('finish', () => {
console.log('All files processed successfully.')
resolve(result)
})

extract.on('error', (error) => {
reject(new Error(`Error during extraction: ${error}`))
console.error('Error during tar extraction:', error.message)
reject(new Error('Failed to extract tar file.'))
})

tarStream.on('error', (error) => {
console.error('Error in tar stream:', error.message)
reject(new Error('Failed to process tar stream.'))
})

// Pipe the ReadableStream into the extractor
tarStream.pipe(extract)
})
}
Loading