项目日志 学成在线 企业级微服务大项目实战《学成在线》【四】(媒资管理模块) 何平安 2024-02-04 2025-03-08 !!!移步我的老博客:企业级微服务大项目实战《学成在线》【四】(媒资管理模块) - 何平安 - 博客园 !!!
封面为啥要用苍穹外卖,想纪念下下以前的项目,不知道现在还跑得起来不哈哈哈哈~
上传图片 大部分都是源文档的东西,懒得写了~
流程:
课程图片上传至分布式文件系统,在课程信息中保存课程图片路径,如下流程:
1、前端进入上传图片界面
2、上传图片,请求媒资管理服务。
3、媒资管理服务将图片文件存储在MinIO。
4、媒资管理记录文件信息到数据库。
5、保存课程信息,在内容管理数据库保存图片地址。
环境准备 首先在minio配置bucket,bucket名称为:mediafiles,并设置bucket的权限为公开。
在nacos配置中minio的相关信息,进入media-service-dev.yaml:
1 2 3 4 5 6 7 minio: endpoint: http://localhost:9000 accessKey: minioadmin secretKey: minioadmin bucket: files: mediafiles videofiles: video
在media-service工程编写minio的配置类:
MinIO配置属性类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 package com.xuecheng.media.config;import org.springframework.boot.context.properties.ConfigurationProperties;import org.springframework.context.annotation.Configuration;@Configuration @ConfigurationProperties(prefix = "minio") public class MinIOProperties {}
Minio配置类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 package com.xuecheng.media.config;import io.minio.MinioClient;import lombok.RequiredArgsConstructor;import org.springframework.boot.context.properties.EnableConfigurationProperties;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;@Configuration @EnableConfigurationProperties({MinIOProperties.class}) @RequiredArgsConstructor public class MinIOConfig { private final MinIOProperties minIOProperties; @Bean public MinioClient minioClient () { return MinioClient.builder() .endpoint("http://localhost:9000" ) .credentials("minioadmin" , "minioadmin" ) .build(); } }
接口定义 根据需求分析,下边进行接口定义,此接口定义为一个通用的上传文件接口,可以上传图片或其它文件。
首先分析接口:
请求地址:/media/upload/coursefile
请求参数:
Content-Type: multipart/form-data;boundary=…..
FormData: filedata=??
响应参数:文件信息,如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 { "id" : "a16da7a132559daf9e1193166b3e7f52" , "companyId" : 1232141425 , "companyName" : null , "filename" : "1.jpg" , "fileType" : "001001" , "tags" : "" , "bucket" : "/testbucket/2022/09/12/a16da7a132559daf9e1193166b3e7f52.jpg" , "fileId" : "a16da7a132559daf9e1193166b3e7f52" , "url" : "/testbucket/2022/09/12/a16da7a132559daf9e1193166b3e7f52.jpg" , "timelength" : null , "username" : null , "createDate" : "2022-09-12T21:57:18" , "changeDate" : null , "status" : "1" , "remark" : "" , "auditStatus" : null , "auditMind" : null , "fileSize" : 248329 }
在media-model定义上传响应模型类:
1 2 3 4 5 6 7 8 9 10 11 12 package com.xuecheng.media.model.dto;import com.xuecheng.media.model.po.MediaFiles;public class UploadFileResultDto extends MediaFiles {}
定义接口如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 /** * @param upload 表单数据 * @param folder 文件夹名-非必须 * @param objectName 文件名-非必须 * @return com.xuecheng.media.model.dto.UploadFileResultDto * @description 文件上传 * @author: woldier * @date: 2023/3/9 22:09 */ @ApiOperation("文件上传") @RequestMapping(value = "/upload/coursefile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE) // 定义请求url与可消费的文件操作content-type类型 public UploadFileResultDto upload( @RequestPart("filedata") MultipartFile upload, @RequestParam(value = "folder", required = false) String folder, @RequestParam(value = "objectName", required = false) String objectName) { return null; }
环境准备 首先在minio配置bucket,bucket名称为:mediafiles,并设置bucket的权限为公开。
在nacos配置中minio的相关信息,进入media-service-dev.yaml:
1 2 3 4 5 6 7 minio: endpoint: http://localhost:9000 accessKey: minioadmin secretKey: minioadmin bucket: files: mediafiles videofiles: video
在media-service工程编写minio的配置类:
MinIO配置属性类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 package com.xuecheng.media.config;import org.springframework.boot.context.properties.ConfigurationProperties;import org.springframework.context.annotation.Configuration;@Configuration @ConfigurationProperties(prefix = "minio") public class MinIOProperties {}
Minio配置类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 package com.xuecheng.media.config;import io.minio.MinioClient;import lombok.RequiredArgsConstructor;import org.springframework.boot.context.properties.EnableConfigurationProperties;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;@Configuration @EnableConfigurationProperties({MinIOProperties.class}) @RequiredArgsConstructor public class MinIOConfig { private final MinIOProperties minIOProperties; @Bean public MinioClient minioClient () { return MinioClient.builder() .endpoint("http://localhost:9000" ) .credentials("minioadmin" , "minioadmin" ) .build(); } }
接口定义 根据需求分析,下边进行接口定义,此接口定义为一个通用的上传文件接口,可以上传图片或其它文件。
首先分析接口:
请求地址:/media/upload/coursefile
请求参数:
Content-Type: multipart/form-data;boundary=…..
FormData: filedata=??
响应参数:文件信息,如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 { "id" : "a16da7a132559daf9e1193166b3e7f52" , "companyId" : 1232141425 , "companyName" : null , "filename" : "1.jpg" , "fileType" : "001001" , "tags" : "" , "bucket" : "/testbucket/2022/09/12/a16da7a132559daf9e1193166b3e7f52.jpg" , "fileId" : "a16da7a132559daf9e1193166b3e7f52" , "url" : "/testbucket/2022/09/12/a16da7a132559daf9e1193166b3e7f52.jpg" , "timelength" : null , "username" : null , "createDate" : "2022-09-12T21:57:18" , "changeDate" : null , "status" : "1" , "remark" : "" , "auditStatus" : null , "auditMind" : null , "fileSize" : 248329 }
在media-model定义上传响应模型类:
1 2 3 4 5 6 7 8 9 10 11 12 package com.xuecheng.media.model.dto;import com.xuecheng.media.model.po.MediaFiles;public class UploadFileResultDto extends MediaFiles {}
定义接口如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @ApiOperation("文件上传") @RequestMapping(value = "/upload/coursefile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE) public UploadFileResultDto upload ( @RequestPart("filedata") MultipartFile upload, @RequestParam(value = "folder", required = false) String folder, @RequestParam(value = "objectName", required = false) String objectName) { return null ; }
controller层中我们需要将文件暂存在本地,让后将临时文件的地址放进服务层方法参数中
接口开发 DAO开发
根据需求分析DAO层实现向media_files表插入一条记录,使用media_files表生成的mapper即可。
Service开发
为了使代码更具有可读性,我们创建了两个枚举工具类,用于区分数据库字段值
可以看到我们操作mediaFile表时需要用到这两种字段,因此我们使用枚举简化
在meida-model的dto包下创建如下两个枚举类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 package com.xuecheng.media.model.dto;public enum MediaResourceType { IMAGE("001001" ,"图片" ), VIDEO("001002" ,"视频" ), OTHER("001003" ,"其它" ) ; private String code; private String description; MediaResourceType(String code, String description) { this .code = code; this .description = description; } public String getCode () { return code; } }
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 package com.xuecheng.media.model.dto;public enum MediaAuditStatus { NOT_Approved("002001" ,"审核未通过" ), Not_Audited("002002" ,"未审核" ), Approved("002003" ,"审核通过" ); private String code; private String description; MediaAuditStatus(String code, String description){ this .code = code; this .description = description; } public String getCode () { return code; } }
除此之外由于我们操作的数据表有公共字段,updateTime,createTime。因此我们可以加入一个mp的自动填充功能。
在media-service的config包下创建如下类
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 package com.xuecheng.media.config;import com.baomidou.mybatisplus.annotation.DbType;import com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor;import com.baomidou.mybatisplus.extension.plugins.inner.PaginationInnerInterceptor;import org.mybatis.spring.annotation.MapperScan;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;@Configuration @MapperScan("com.xuecheng.media.mapper") public class MybatisPlusConfig { @Bean public MybatisPlusInterceptor mybatisPlusInterceptor () { MybatisPlusInterceptor interceptor = new MybatisPlusInterceptor (); interceptor.addInnerInterceptor(new PaginationInnerInterceptor (DbType.MYSQL)); return interceptor; } }
Service方法需要提供一个更加通用的保存文件的方法。
定义请求参数类:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 package com.xuecheng.media.model.dto;import lombok.Data;@Data public class UploadFileParamsDto { private String filename; private String contentType; private String fileType; private Long fileSize; private String tags; private String username; private String remark; }
定义service方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 com.xuecheng.media.service.MediaFileService UploadFileResultDto uploadFile (Long companyId, UploadFileParamsDto uploadFileParamsDto, String LocalFilePath) throws XueChengPlusException; MediaFiles insertMediaFile2DB (Long companyId, UploadFileParamsDto uploadFileParamsDto, String md5, String bucket,String objectName) ;
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 com.xuecheng.media.service.impl.MediaFileServiceImpl @Override public UploadFileResultDto uploadFile (Long companyId, UploadFileParamsDto uploadFileParamsDto, String localFilePath) throws XueChengPlusException { String mimeType = getMimeType(uploadFileParamsDto.getFilename()); String basePath = LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy/MM/dd/" )); int index = uploadFileParamsDto.getFilename().lastIndexOf("." ); String fileSuffix = uploadFileParamsDto.getFilename().substring(index); String md5 = getMd5(localFilePath); String objectName = basePath + md5 + fileSuffix; boolean minIOUpload = minIOUpload(localFilePath, mimeType, fileBucket, objectName); if (!minIOUpload) XueChengPlusException.cast("MinIO上传出错" ); MediaFiles files = insertMediaFile2DB(companyId, uploadFileParamsDto, md5, fileBucket, objectName); if (files == null ) XueChengPlusException.cast("文件上传后保存信息到数据库失败" ); UploadFileResultDto uploadFileResultDto = new UploadFileResultDto (); BeanUtils.copyProperties(files, uploadFileResultDto); return uploadFileResultDto; } @Transactional public MediaFiles insertMediaFile2DB (Long companyId, UploadFileParamsDto uploadFileParamsDto, String md5, String bucket, String objectName) { MediaFiles files = mediaFilesMapper.selectById(md5); if (files == null ) { MediaFiles mediaFiles = new MediaFiles (); BeanUtils.copyProperties(uploadFileParamsDto, mediaFiles); mediaFiles.setId(md5); mediaFiles.setCompanyId(companyId); mediaFiles.setBucket(bucket); mediaFiles.setFilePath(objectName); mediaFiles.setFileId(md5); mediaFiles.setUrl("/" + bucket + "/" + objectName); mediaFiles.setStatus("1" ); mediaFiles.setAuditStatus(MediaAuditStatus.Approved.getCode()); int insert = mediaFilesMapper.insert(mediaFiles); if (insert <= 0 ) { log.debug("向数据库保存文件失败,bucket:{},objectName{}" , fileBucket, objectName); return null ; } return mediaFiles; } return files; } @NotNull private static String getMd5 (String localFilePath) throws XueChengPlusException { String md5 = null ; try { md5 = DigestUtils.md5Hex(Files.newInputStream(new File (localFilePath).toPath())); } catch (IOException e) { XueChengPlusException.cast("md5计算时出错" ); } return md5; } private String getMimeType (String fileName) { if (fileName == null ) fileName = "" ; ContentInfo contentInfo = ContentInfoUtil.findExtensionMatch(fileName); String mimeType = MediaType.APPLICATION_OCTET_STREAM_VALUE; if (contentInfo != null ) mimeType = contentInfo.getMimeType(); return mimeType; } private boolean minIOUpload (String localFilePath, String fileType, String bucket, String objectName) { try { minioClient.uploadObject( UploadObjectArgs.builder() .bucket(bucket) .object(objectName) .filename(localFilePath) .contentType(fileType) .build() ); } catch (Exception e) { log.error("文件上传到MinIO出错,buckcet:{},path:{},error:{}" , bucket, objectName, e.getMessage()); e.printStackTrace(); return false ; } return true ; }
接口代码完善 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 @ApiOperation("文件上传") @RequestMapping(value = "/upload/coursefile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE) public UploadFileResultDto upload ( @RequestPart("filedata") MultipartFile upload, @RequestParam(value = "folder", required = false) String folder, @RequestParam(value = "objectName", required = false) String objectName) throws IOException, XueChengPlusException { File tempFile = File.createTempFile("minio" , "temp" ); upload.transferTo(tempFile); String absolutePath = tempFile.getAbsolutePath(); UploadFileParamsDto uploadFileParamsDto = new UploadFileParamsDto (); uploadFileParamsDto.setFilename(upload.getOriginalFilename()); uploadFileParamsDto.setFileSize(upload.getSize()); uploadFileParamsDto.setFileType(MediaResourceType.IMAGE.getCode()); Long companyId = 123456789L ; UploadFileResultDto uploadFileResultDto = mediaFileService.uploadFile(companyId, uploadFileParamsDto, absolutePath); tempFile.deleteOnExit(); return uploadFileResultDto;
service事务代码优化 上边的service方法优化后并测试通过,现在思考关于uploadFile方法的是否应该开启事务。
目前是在uploadFile方法上添加@Transactional,当调用uploadFile方法前会开启数据库事务,如果上传文件过程时间较长那么数据库的事务持续时间就会变长,这样数据库链接释放就慢,最终导致数据库链接不够用。
我们只将addMediaFilesToDb方法添加事务控制即可,uploadFile方法上的@Transactional注解去掉。(上小节代码已经时这样做的)
但是现在的问题是,controller调用的service方法upload没用加入事务注解,相当于在service中一个没有事务的方法调用了另一个事务方法,事务不生效
下边分析原因:
如果在uploadFile方法上添加@Transactional注解,代理对象执行此方法前会开启事务,如下图:
如果在uploadFile方法上没有@Transactional注解,代理对象执行此方法前
不进行事务控制,如下图:
现在在addMediaFilesToDb方法上添加@Transactional注解,也不会进行事务是因为并不是通过代理对象执行的addMediaFilesToDb方法。为了判断在uploadFile方法中去调用addMediaFilesToDb方法是否是通过代理对象去调用,我们可以打断点跟踪。
我们发现在uploadFile方法中去调用addMediaFilesToDb方法不是通过代理对象去调用。
如何解决呢?通过代理对象去调用addMediaFilesToDb方法即可解决。
我们先获取代理对象,然后调用代理对象的insertMediaFile2DB方法
1 2 MediaFileService proxy = (MediaFileService)AopContext.currentProxy();MediaFiles files = proxy.insertMediaFile2DB(companyId, uploadFileParamsDto, md5, fileBucket,objectName);
但是只修改这个代码启动会报错
1 Cannot find current proxy: Set 'exposeProxy' property on Advised to 'true' to make it available, and ensure that AopContext.currentProxy() is invoked in the same thread as the AOP invocation context.
我们需要在启动类下加入注解并且设置exposeProxy属性,除此之外可能的报错原因是没有加入包
1 @EnableAspectJAutoProxy(exposeProxy = true)
1 2 3 4 <dependency > <groupId > org.aspectj</groupId > <artifactId > aspectjweaver</artifactId > </dependency >
我们可以在代码中插入数据库后除0模拟错误,看一下是否进行了数据库回归,经过测试可以发现进行了回滚
接口测试 1 2 3 4 5 6 7 8 9 POST {{gateway_host}}/media//upload/coursefile Content-Type: multipart/form-data; boundary=WebAppBoundary --WebAppBoundary Content-Disposition: form-data;name="filedata" ; filename="123.jpg" Content-Type: application/octet-stream < d:/123.jpg
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 POST http: HTTP/1.1 200 OK transfer-encoding: chunked Content-Type: application/json Date: Fri, 10 Mar 2023 08 : 33 : 46 GMT { "id" : "8a58662af30ace3e83f629a10ddd8662" , "companyId" : 123456789 , "companyName" : null , "filename" : "1.jpg" , "fileType" : "001001" , "tags" : null , "bucket" : "mediafiles" , "filePath" : "2023/03/10/8a58662af30ace3e83f629a10ddd8662.jpg" , "fileId" : "8a58662af30ace3e83f629a10ddd8662" , "url" : "/mediafiles/2023/03/10/8a58662af30ace3e83f629a10ddd8662.jpg" , "username" : null , "createDate" : null , "changeDate" : null , "status" : "1" , "remark" : null , "auditStatus" : "002003" , "auditMind" : null , "fileSize" : 9778 }
上传视频